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ou  are  now  well  prepared  tor 
the  rough  and  tumble  of 
bureaucratic  politics,  having 
survived  5  months  of  aca¬ 
demic  politics.  I  am  sure  you 
can  now  appreciate  C  P  Snow's  quip 
that  academic  politics  are  most  vicious, 
but  the  stakes  are  so  low, 

1  know  you  are  anxious  to  get  on 
with  your  assignments,  where  the 
stakes  are  not  at  all  low,  and  to  put  to 
work  all  you  have  learned  here:  to 
leave  behind  long  and  rigorous  days  of 
learning  for  the  normal  life  of  a  pro¬ 
gram  manager:  to  relax  with  the 
70-hour  work  weeks  that  are  the  norm 
for  the  DOD  program  manager;  to 
leave  behind  nights  spent  in  the  library 
poring  over  textbooks  in  favor  of  a 
pleasant  evening  perusing  a  mere  five 
pounds  of  contract  documents  and  a 
three-foot-high  stack  of  briefing  charts. 

1  am  sure  you  are  anxious  to  meet 
the  people  you  will  work  with  — 
auditors,  inspectors,  investigators, 
congressional  staffers,  headquarters 
people,  and  others  who  will  arrive 
with  a  smile  and  a  cheerful  greeting: 
"Hi.  we  are  here  to  help  you." 

I  imagine  you  are  eager  to  test  your 
program  manager  skills  — to  find  out 
how  good  you  are  at  writing 
viewgraphs."  and  how  deft  at  mak¬ 
ing  last-minute  changes  that  are  the 
lifeblood  of  the  management-by- 
viewgraph  crowd. 

You  have  all  these  things  to  look  for¬ 
ward  to.  countless  frustrations  and 
myriad  problems  — problems  even  the 
Defense  Systems  Management  College 
did  not  prepare  you  tor.  That  is  the  im- 
menselv  challenging  and  difficult  role 
of  the  program  manager.  It  there  is  any 
lob  in  our  government  that  makes  a 
direct  and  sizeable  contribution  to  the 
detense  ot  this  nation  this  is  it. 

Better  than  most,  you  know  our 
detense  is  underwritten  by  the  quality 
ot  the  uvupou  systems  we  provide  to 
our  combat  forces.  We  rely  on  our 
ability  to  equip  America  s  defense 
forces  with  the  high-quality  militarv 
systems  needed  to  deter  a  numerically 
superior  adversary  You  have  been 
chosen  to  share  that  enormous  respon- 
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;  sibility.  You  have 
been  selected  for 
positions  that  will  allow  you  to  in 
fluence  directly  the  efficiency  of  our  ac 
quisition  system  and  the  weapons  we 
acquire. 

Today,  as  you  graduate  from  the 
j  school  that  Dave  Packard  set  up  some 
15  years  ago.  the  only  college  of  its 
kind  in  the  Free  World.  I  want  to  talk 
a  little  about  being  a  program 
manager  — about  the  challenge  ahead 
and  the  nature  of  the  business  you  are 
about  to  enter. 

At  the  Defense  Systems  Manage¬ 
ment  College,  you  have  not  just 
studied  management.  You  have  ex¬ 
amined  leadership  and  developed  an 
understanding  of  the  importance  of 
both  functions  to  the  success  of  the 
defense  acquisitions  system.  The 
]  Defense  Systems  Management  College 
teaches  that  professional  acquisition 
managers  must  appreciate  both 
disciplines  and  the  qualitites  associated 
with  each.  Program  managers  must 
J  manage  things  and  lead  people. 

Some  people  have  not  understood 
the  importance  of  both  functions  to  the 
defense  acquisition  system.  According 
to  these  arm-chair  strategists,  the 
military's  emphasis  on  management 
,  was  responsible  for  all  problems 
'  throughout  our  defense  establishment 
whether  real  or  merely  perceived. 

Within  acquisition,  there  were  prob¬ 
lems.  The  indefensible  prices  that  a  feu- 
companies  were  proposing  to  charge, 
and  that  we  found  when  we  came, 
reinforced  the  criticism  of  the  so-c ailed 
reformers  No  one  seemed  to 
remember  that  ire  had  found  those 
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prices  and  refused  to  pay  them.  Our 
management  has  been  dissected  and 
examined  in  infinite  detail.  Every  ma¬ 
jor  "think  tank"  has  cashed-in  on  the 
trendy  demand  for  studies  of  our  ac¬ 
quisition  management,  which  really 
means  thev  simply  joined  the  attack. 
We  now  have  results  of  the  President  s 
Blue  Ribbon  Commission  on  Defense 
Management,  a  host  of  internal  DOD 
studies,  and  legislation  that  has  come 
so  quickly  that  we  could  hardly  imple¬ 
ment  one  law  before  the  C  ongress  was 
writing  another. 

We  should  not  minimize  the  in¬ 
herent  and  historical  problems  in  ac¬ 
quisition  management  In  tact  I 
directed  a  major  review  and  reform  of 
acquisition  practices  shortly  after  this 
.•Administration  took  office  It  was 
i  lear  that  our  managers  the  men  and 
women  who  manage  our  acquisition 


system  — were  not  the  problem.  It  is 
reassuring  to  note  that  all  of  the  studies 
of  our  acquisition  system  reaffirmed 
the  skill  and  dedication  of  our  acquisi¬ 
tion  people. 

1  have  told  everyone  who  will  listen 
that  it  is  the  dedicated,  professional  ac¬ 
quisition  managers  who  found  and 
fixed  most  of  the  problems  in  our  ac¬ 
quisition  system.  A  brief  review  of 
what  are  now  popularly  called 
"reforms"  testifies  to  the  quality  of 
leadership  of  the  acquisition 
community. 

Much  of  what  we  are  trying  to  do 
today  is  to  manage  "things"  so  that  our 
leaders  can  operate  more  freely  and  ef¬ 
fectively.  The  "enterprise”  programs, 
acquisition  streamlining,  model  in¬ 
stallations,  professional  education  for 
acquisition  personnel,  show  we  believe 
that  the  real  solutions  to  acquisition 
problems  are  to  remove  barriers  to 
creativity  and  to  foster  an  environment 
of  encouragement  and  trust. 

We  have  adopted  many  other  in¬ 
itiatives  to  give  our  people  more  effec¬ 
tive  tools.  Among  these  are:  multi-year 
procurement;  baselined  acquisition 
programs:  a  focus  on  quality  that  goes 
beyond  performance  to  include  the 
whole  range  of  factors  that  determine 
the  value  of  our  hardware  to  the 
soldier,  sailor,  and  airman  in  the  field; 
reduction  of  military  specifications; 
off-the-shelf  procurement:  cooperative 
R&D  with  allies;  2-year  budgeting; 
contractor  self-disclosure  programs; 
use  of  suspension  and  debarment  pro¬ 
ceedings:  and  the  most  extensive  and 
pervasive  audit  of  defense  programs 
ever  undertaken. 

These  are  correctly  cited  as  the  solu¬ 
tions  to  our  management  problems  of 
the  past.  But  what  is  not  heralded, 
what  remains  virtually  unknown  is 
that  these  dynamic,  cost-reducing  pro¬ 
grams  were  not  the  creation  of  the 
gurus"  in  the  think-tanks  or  the  prod¬ 
ucts  ot  rounds  of  congressional  hear¬ 
ings.  but  ot  the  vision  and  profes¬ 
sionalism  ot  our  ac  quisition  managers. 

Baselining  tor  example  is  rooted  in 
an  Air  Force  Systems  Command  in¬ 
itiative  dating  trom  1  °82  In  tact,  some 
correctly  credit  the  [’resident  with 
establishing  the  first  baseline  -on  the 
B-l  bomber  Two-year  budgeting  is 
another  recentlv  discovered  idea  but 
it  has  been  in  mv  speeches  and 
testimonv  since  earlv  in  mv  tenure  as 
Secrctarv  ot  Defense.  I 
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Clearly,  the  desire  for  excellence  that 
underpinned  the  dynamic  im¬ 
provements  underway  within  the 
defense  acquisition  community  are  not 
indicative  of  impoverished  leadership 
or  a  self-serving  bureaucracy.  The 
defense  acquisition  system,  despite 
historic  problems  and  the  need  for 
reorganization  now  underway,  is  run 
bv  extradordinarily  dedicated  and 
capable  people.  I  particularly  want  to 
call  your  attention  to  the  extraordi¬ 
narily  able  record  of  our  new  Under¬ 
secretary  for  Acquisition  who  is  in 
charge  of  the  whole  acquisition 
process. 

1  will  not  pretend  that  we  did  not 
have  problems  in  acquisition.  I  am  sure 
some  problems  will  always  be  with  us. 
But  the  problem  was  not  with  the 
dedication  and  quality  of  our  people. 

As  you  graduate  from  the  Defense 
Systems  Management  College  today, 

I  want  to  remind  you  of  this.  In  join¬ 
ing  or  returning  to  the  defense  acquisi¬ 
tion  management  team,  you  will 
become  part  of  a  highly-regarded 
group  of  professionals.  I  am  one  of 
those  whose  high  regard  you  have. 

There  is  no  negative  connotation  to 
the  title  of  manager  in  acquisition. 
Rather,  we  properly  understand  that 
a  manager  s  duty  is  an  expression  of 
leadership  responsiblility. 

In  America's  military,  we  value  the 
successful  leader  above  all  else.  And 
the  defense  acquisition  system  is  no 
different.  The  real  job  ot  the  program 
manager  is  no  different  than  that  of 
any  other  officer  of  the  government, 
whether  a  civilian  or  uniformed 
officer .  The  job  is  the  same:  to  lead. 
Your  responsibility  as  acquisition 
managers  is  to  lead  a  diverse  team  ot 
professional  men  and  women  in  ac¬ 
complishing  your  portion  of  the 
defense  mission  which  is  providing 
our  armed  forces  with  high-quality 
military  equipment . 

I  eadership  has  as  much  bearing  on 
the  sui  cess  of  the  program  manager  as 
it  ( ioi's  on  the  sue  cess  of  a  c  ombat  c  orn- 
m.inder.  So  vour  challenge  is  to  be  a 
leader  to  mark  the  proper  path  tor 
vour  subordinates,  colleagues  and,  in 
\  our  case  contractors.  Your  challenge 
ic  to  attain  the  verv  highest  standards 
of  ixellenie  and  to  demonstrate  an 
unwavering  commitment  to  absolute 
^  integrity 

/’’  -eo in:  .Mi/iiao  ' 


These  two  qualities  of  leadership 
will  determine  your  success  as  a  pro¬ 
gram  manager.  Absolute  and  un¬ 
bridled  commitment  to  excellence  and 
integrity  are  the  ingredients  for 
success. 

As  leaders  of  the  acquisition  arm  of 
defense,  you  must  be  just  as  commit¬ 
ted  to  leading  by  example  as  is  the 
Wing  Commander  who  flies  lead  in  the 
first  wave,  the  Battalion  Commander 
at  the  front  line,  or  the  Ship's  Captain 
on  the  bridge.  The  very  high  standards 
you  demand  of  yourselves  as  of  all 
military  officers  and  public  servants 
are  the  standards  that  must  be  emula¬ 
ted  within  the  acquisition  community. 

I  hope  you  can  approach  your  ac¬ 
quisition  duties  in  this  way.  There  is 
no  secret  to  success  as  a  program 
manager— whether  you  are  an  officer, 
civilian,  or  member  of  industry.  Each 
of  us  must  excel,  must  lead,  and  must 
be  above  reproach. 

Your  daily  activities,  though 
couched  in  the  lexicon  of  the 
businessman,  must  be  governed  by  the 
creed  and  dedication  of  soldiers  and 
officers. 

Your  duty  and  mission  is  quite 
simply  the  acquisition  of  the  combat 
capabilities  that  will  keep  this  nation 
free.  Your  honor  will  be  your  most 
prized  possession.  It  will  sustain  you 
in  the  severe  tests  ahead  and  serve  as 
the  foundation  for  unyielding  integ¬ 
rity,  the  hallmark  of  the  defense  ac¬ 
quisition  manager.  Your  daily  ac¬ 
tivities  must  demonstrate  the  urgency 
of  the  nation's  defense  today,  and  your 
responsibility  for  the  defense  of 
tomorrow. 

This  is  a  time  of  great  challenge  for 
the  defense  acquisition  community. 
Never  before  has  there  been  a  greater 
need  tor  good  ideas,  nor  an  acquisition 
system  more  responsive  to  those  ideas. 
Our  reorganization  of  defense  acquisi¬ 
tion.  our  appointment  of  a  highlv-qual- 
itied  man  to  be  in  charge  of  the  whole 
system,  have  eliminated  barriers  be¬ 
tween  program  managers  and  decision¬ 
makers  and  given  you  the  leverage 
needed  to  put  good  ideas  into  action. 

Today,  vour  creative  management 
is  needed  even  more  than  before.  Our 
F  ist  a  I  1°87  defense  budget  is  7  percent 
in  real  terms  below  the  fiscal  vear  1°85 
figure  and  the  t  ongress  is  still  slashing 
away  .it  the  f  iscal  1°88  budget  Flu- 
best  efforts  ot  the  acquisition  com¬ 


munity  are  urgently  needed  to  ensure 
the  American  people  receive  the  fullest 
measure  of  defense  from  every  acquisi¬ 
tion  dollar  spent.  To  do  less  will  com¬ 
pound  the  risk  our  nation  faces  in  a 
world  hostile  to  freedom. 

You  will  be  severely  tested  to  main¬ 
tain  the  quality,  stability,  and  afford¬ 
ability  of  your  programs  in  the  years 
ahead.  You  will  have  a  central  role  in 
our  effort  to  maintain  the  momentum 
of  defense  modernization. 

In  the  last  5  months,  the  Defense 
Systems  Management  College  has 
attempted  to  prepare  you  for  the  very 
difficult  years  ahead.  I  have  no  doubt 
it  has  given  you  a  sound  foundation 
for  success  in  the  future.  Your  relation¬ 
ship  with  DSMC  must  not  end  today. 
This  College  can  and  will  continue  to 
contribute  to  your  individual  success 
and  the  quality  of  our  acquisition 
system.  Through  the  College  regional 
centers  and  other  programs,  your 
education  will  continue.  Perhaps  even 
more  importantly,  the  resources  of  this 
outstanding  institution  will  be 
available  to  you.  The  College's 
expanding  research  efforts  and  the  con¬ 
siderable  talent  of  the  faculty  and  staff 
are  committed  to  assist  in  your  pursuit 
of  excellence  in  defense  acquisition.  I 
urge  you  to  take  advantage  of  those 
resources,  just  as  1  urge  the  College  to 
continue  building  its  outreach  pro¬ 
grams  in  all  acquisition  activities.  The 
challenge  ahead  is  so  great  and  the 
stakes  so  high  that  we  cannot  afford 
to  ignore  this  avenue  for  further 
improvement 

1  hope  this  graduating  class  and  the 
faculty  and  staff  of  the  College  will 
build  on  your  accomplishments. 
Defense  acquisition  has  made  progress 
in  its  effort  to  improve  America's 
defenses,  but  we  must  go  even 
farther  —  and  we  must  do  so  together. 

Let  me  congratulate  you  all.  Your 
graduation  from  DSMC  certifies  your 
readiness  for  the  most  difficult  posi¬ 
tions  within  the  department's  acquisi¬ 
tion  community  and  marks  you  tor 
even  greater  challenges.  You,  and  your 
families,  should  be  very  proud  ot  this 
accomplishment ,  It  is  my  privilege,  as 
I  know  it  is  Dick  Godwins  to  con¬ 
gratulate  you  on  your  graduation 
today.  ■ 

"A  man's  judgment  is  no  better  than 
his  information. 

Author  Unknown. 
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A  survey  course  on  procurement 
for  program  office  personnel. 


This  one-week  course  is  designed 
to  provide  an  overall  understand¬ 
ing  of  the  systems  acquisition  con¬ 
tracting  process  from  planning  for 
a  solicitation  through  contract 
closeout.  It  concentrates  on  key 
activities  required  to  award  and 
administer  a  government  con¬ 
tract.  Included  are  such  topics  as 
program  manager/contracting 
officer  relationships,  acquisition 
planning,  structuring  contracts, 
request  for  proposal  preparation, 
cost  proposal  evaluation,  source 
selection,  modifications,  DOD 
cost  principles,  contractor  profit, 
subcontract  management  and 
contract  administration. 


Lecture-discussions  are  punc¬ 
tuated  with  short  case  studies  to 
reinforce  student  learning. 
CMPMC  has  been  designed  pri¬ 
marily  for  DOD  personnel  working 
in  a  program  management  office 
or  related  supporting  activity.  Mili¬ 
tary  personnel  in  grades  0-2  and 
above,  and  DOD  civilians  in 
grades  GS-09  and  above  are  the 
intended  audience.  Individuals 
with  similar  positions  in  other 
federal  agencies  or  the  defense 
industry  are  also  encouraged  to 
attend. 

Point  of  Contact: 

Frank  Meneely 
(703)  664-6683 
AV  354-6683 
Defense  Systems 
Management  College 
Fort  Belvoir.  VA  22060-5426 
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long  with  increased  em¬ 
phasis  on  defense  has  come 
an  increased  emphasis  on 
keeping  defense  costs  down. 
Control  of  escalating  cost  of 
government  developments  and 
elimination  of  cost  overruns  are  prime 
concerns  of  the  program  managers 
(PMs)  for  system,  subsystem,  and 
component  developments  during  all 
phases  of  the  acquisition  process.  One 
of  the  prime  considerations  in  controll¬ 
ing  cost  growth  is  letting  contracts  in¬ 
itially  at  realistic  target  cost  with 
known  cost  risk.  Therefore,  evaluating 
proposals  for  cost  realism  becomes  an 
important  part  of  the  cost-control 
process.  The  task  of  evaluating  pro¬ 
posals  for  cost  realism  may  be  more 
difficult  for  early  development 
phases — proof  of  principal,  concept 
development,  technology  valida¬ 
tion— than  later  phases,  due  to  sub¬ 
stantial  technology  uncertainty  and 
program  risk.  However,  the  approach 
proposed  in  this  paper  would  apply  to 
any  phase.  It  was  developed  and  uti¬ 
lized  during  the  Exoat  mospheric 
Reentry-Vehicle  Interceptor  Subsystem 
lERIS)  Source  Selection  Evaluation 
Board  (SSEB)  selection  process  for  the 
Functional  Technology  Validation 
(FTVi  contract  award. 

The  Problem 

This  paper  is  about  the  source  selec¬ 
tion  cost  evaluation  process  for  cost 
reimbursable  contracts  with  particular 
emphasis  on  the  evaluation  process  for 
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measuring  cost  realism  of  competitive 
proposals.  The  objective  is  to  assure 
more  realistic  contractor  cost  pro¬ 
posals  and  contract  awards  at  more 
realistic  target  costs.  The  problem  is 
what  to  use  as  the  standard  against 
which  the  cost  realism  of  the  proposals 
is  measured  and  how  to  measure  cost 
realism. 


Traditional  Approach 

The  traditional  approach  in  request 
for  proposals  (RFPs)  and  SSEB  pro¬ 
posal  evaluations  require  offerors  to 
provide  a  cost  proposal  with  a  target 
cost  and  proposed  fee  arrangement. 
Therefore,  his  proposal  is  a  proposed 
program,  and  a  proposed  schedule  and 
a  proposed  target  cost.  Then,  usually 
the  SSEB  has  a  government  point 
estimate  of  what  the  effort  should  cost 
Now  the  simplest  approach  to  measur 
ing  cost  realism  would  be  to  compare 


the  proposed  target  cost  with  the 
government  estimate.  In  this  paper,  I 
suggest  that  this  is  not  a  very  good  ap¬ 
proach  and  propose  that  a  different 
method  he  utilized. 

Shortcomings  of  Traditional 
Approach 

The  traditional  SSEB  approach  has 
several  problems. 

—  All  proposals  in  response  to  a 
specific  request  for  proposal  are  unique 
and  therefore  should  not  be  compared 
to  a  single  government  generic  cost 
estimate. 

—  It  is  not  known  where  the  proposed 
target  cost  number  fits  relative  to  a 
probability  of  success  number;  i.e., 
how  optimistic  or  pessimistic  is  the 
proposed  target  cost?  Likewise,  it  is  not 
known  how  much  overrun  potential 
exists  in  the  proposed  program. 

—  It  is  not  known  where  the  govern¬ 
ment  point  estimate  fits  relative  to  a 
probability  of  success  number. 

Proposed  Method 

The  first  task  is  to  define  cost 
realism.  There  is  no  standard  definition 
that  tits  all  cases;  therefore,  one  of  the 
first  tasks  of  RFP  preparation  should 
be  the  definition.  The  definition  should 
convey  to  the  offerors  how  cost 
realism  would  be  measured  by  the 
SSEB  (and  scored  if  it  is  to  be  scored). 

I  believe  cost  realism  can  only 
k  be  measured  against  a  standard 
m  where  cost  estimates  are  developed 
H  on  a  risk 


model.  Furthermore,  I  propose  that  to 
make  the  process  more  effective,  the 
offeror's  proposal  should  be  presented 
on  a  risk  model  rather  than  the  current 
method  which  is  only  a  point  estimate 
without  any  knowledge  of  the  point 
estimate's  risk  (probability  of  being  ex¬ 
ceed.  d).  Therefore,  here  are  some 
recommendations. 


—The  request  for  proposal  should  re¬ 
quire  all  cost  proposals  to  be  bid  on  a 
risk  basis. 


—  The  SSEB  should  be  required  to 
develop  its  own  cost  estimate  on  a  risk 
basis  as  described  below  for  each  pro¬ 
posal  (there  should  be  a  cost  realism 
standard  developed  for  each  proposal). 

—  The  SSEB  cost  estimate  mean  and 
standard  deviation  and  the  contractor's 
bid  should  be  displayed  on  a  risk  rela¬ 
tionship  (Bell)  curve  as  described 
below. 


How  Is  SSEB  Estimate  Developed 
On  a  Risk  Basis? 


— The  request  for  proposal  should  re¬ 
quire  that  all  cost  proposals  include  a 
cost  rationale  section  where  the  con¬ 
tractor  can  document  (explain)  why 
cost  numbers  are  credible  and 
reasonable,  and  schedule  and  program 
are  credible  and  reasonable.  The  of¬ 
feror  should  be  required  to  identify  in 
this  section  the  program  risk  areas  of 
the  proposal  and  the  risk-reduction 
plans. 


—The  request  for  proposal  should  re¬ 
quire  all  major  subcontractors  to  bid 
on  a  risk  basis  and  that  this  proposal 
be  made  available  to  the  Source  Selec¬ 
tion  Evaluation  Board. 


Before  commencing  this  cost  estima¬ 
tion  process,  the  Source  Selection 
Evaluation  Board,  working  through 
the  contracting  officer,  must  cause 
contractors  to  bring  proposals  in  com¬ 
pliance  with  the  RFP  through  tradi¬ 
tional  SSEB  procedure  (if  the  SSEB 
determines  the  proposal  is  not  in  com¬ 
pliance  with  the  RFP  or  is  unclear). 
Then,  the  first  step  in  the  cost¬ 
estimating  process  is  one  where  the 
proposed  schedule  is  adjusted  by  the 
SSEB  for  reasonableness  (if  it  is  judg¬ 
ed  to  be  unreasonable).  Other  areas 
may  have  to  be  adjusted  if  judged  to 
be  unlikely  to  occur  in  the  fashion  pro¬ 
posed.  In  addition,  the  SSEB  must 
determine  the  schedule  interrelation¬ 
ships  that  are  impacted  in  the  most 
pessimistic  and  most  optimistic  low- 
level  estimates.  Then  a  bottoms-up 
cost  estimate  is  made  on  a  risk  model 
as  described  below.  (There  are  other 
methods  or  models  of  developing  the 
cost  estimate  on  a  risk  basis.) 
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The  procedure  for  the  bottoms-up 
cost  estimates  performed  by  the  Source 
Selection  Evaluation  Board  requires 
that  each  cost  element  (labor  hours, 
travel,  manufacturing,  materials,  etc.) 
for  each  item  of  hardware  or  function 
■  be  estimated  three  ways— most  likely, 
most  pessimistic,  and  most  optimistic. 
Since  these  estimates  are  performed  at 
low  levels  of  the  work  breakdown 
structure  (WBS)  in  the  proposal,  the 
statistical  validity  of  the  bottoms-up 
estimate  is  enhanced  as  the  summation 
process  evolves.  Each  low-level 
estimate  may  then  be  input  to  a  com¬ 
puter  program  (usually  a  spreadsheet 
or  data  base  management  program)  to 
;  obtain  results  at  the  total  proposal 
level  for  the  most  likely,  most 
pessimistic,  and  most  optimistic  point 
estimates  (M,  H,  and  L  below). 

Having  the  above  three  sets  of 
estimates,  a  formula  can  be  used  to 
|  construct  the  '  Bell-Shaped”  cost  rela¬ 
tionship  curve  (see  Figure  1). 

,  The  values  of  X  and  SD.  the  contrac- 
I  tor's  bid  and  the  PMs  budget  are  then 
put  on  a  normal  curve  as  described  (see 
Figure  2  tor  example). 

Application 
and  Interpretation  of 
Risk  Relationship  Curve 
(Figure  2) 

The  curve  invokes  the  central 
limit  theorem.  It  asserts  that  certain 
statistics,  such  as  the  arithmetic  mean, 
tend  to  be  normally  distributed  as  the 
sample  size  becomes  large.  Thus,  it 
samples  are  drawn  from  a  population 
that  is  not  normally  distributed  (skew¬ 
ed),  the  successive  sample  means  will 
form  a  distribution  that  is  approx¬ 
imately  normal. 

—  The  normal  bel]  curve  is  used  to 
reflect  the  mean  (X),  standard  devia¬ 
tion  (SD)  and  make  some  statement 
relative  to  confidence  of  costs.  For  ex¬ 
ample,  the  mean  plus  or  minus  two 
standard  deviations  would  express  a  °5 
percent  confidence  interval  that  the 
cost  value  would  tall  between. 

The  normal  curve  mav  also  be  used 
to  reflect  a  not  to  exceed  statement 
U  umulative  interval  between  zero  and 
an  X  value)  or  to  express  the  idea  ot 
risk  to  the  right  or  left  ot  a  specific 
point  on  the  X  axis.  For  example  at  the 
i  mean  the  relative  value  is  30  50  At 
plus  1  SI)  it  is  84  It?  <84  percent  <  hance 
ot  not  being  exceeded  or  In  percent 
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most  pessimistic  low  level  estimate 
most  likely  low  level  estimate 
most  optimistic  low  level  estimate 
total  pessimistic  point  estimate 
total  most  likely  point  estimate 
total  optimistic  point  estimate 


Then  the  mean  =  X 


H  +  4M  +  L 
6 


Let  the  variance  of  each  low  level  estimate  =  Vj 

Then  Vj  =  Hj  -  Lj  2  for  i  =  1  to  N  where  N  =  number  of  low  level  estimates 

in  the  summation  to 
obtain  total  point 
estimate 

Let  V  =  the  sum  of  all  the  Vj  (i  =  1  to  N) 

Then  the  Standard  Deviation  =  SD  =  V 


chance  ot  being  exceeded).  At  minus 
1  SD  is  lo  84.  For  X  values  not  falling 
on  1,  2.  or  3  SD,  area  under  the  curve 
must  be  calculated.  The  formula  (X- 
X)  divided  by  SD  provides  the  number 
of  standard  deviations  that  X  is  from 
the  mean.  Then,  this  number  is  used 
in  the  standard  statistical  Took  up” 
table  to  obtain  the  area  to  be  added  to 
(or  subtracted  from)  50  percent  to  ob¬ 
tain  the  risk  of  being  exceeded. 

In  summary,  any  X  value  on  the 
curve  (cost  estimate  or  contractor  bid) 
can  be  defined  as  having  a  probability 
ot  being  exceeded  (number  on  right )  or 
having  a  probability  of  being  met 
(number  on  left).  This,  then,  shows  the 
risk  of  the  program  being  accomplish¬ 
ed  at  a  given  target  cost  (X  value). 

— Creating  the  curve  (the  cost  numbers 
are  only  tor  illustration). 

—  SSEB  mean  obtained  from  the  for¬ 
mula  =  S40p\l 

SSEB  SD  obtained  from  the  for¬ 
mula  =  S44M 

1  iori/ont.il  sc  ale  is  determined  bv 
plotting  +  12.3  SI)  values 

SSF.B  point  estimate  < most  likely 
obtained  from  the  cost  estimate 
model  =  $43oM  'This  is  plotted 
to  il lust r.ite  that  i'  -\iv  be  riskv 
to  use  SSF.B  point  t  -'Dilates  as  a 
standard  1 


(All  of  the  following  percentage  risks 
are  calculated  using  formula  and  "look 
up”  table) 

—  Percentage  risk  5  °5  on  bid  ot 
S425M 

—  Percentage  risk  8  02  on  SSEB 
S436M  point  estimate 

—  Percentage  risk  50  50  mean 

—  Percentage  risk  00  1  on  PME 
budget 

—  The  Figure  2  curve  represents  an  ex¬ 
ample  ot  the  SSEB  risk  model  which 
will  be  utilized  to  measure  cost  realism. 
It  includes  the  contractor  -  bid  or  target 
cost,  the  SSEB  point  estimate  the 
measure  of  central  tendency  or  mean 
and  the  measure  ot  dispersion  or  stan¬ 
dard  deviation.  The  mean  and  stan¬ 
dard  deviation  could  be  the  SSEB  stan¬ 
dard  used  to  measure  cost  realism  ot 
the  proposal.  For  illustration  discus¬ 
sion  and  assignment  ot  risk  dollars  the 
project  managers  parametrical  deriv¬ 
ed  Project  Manager  s  Estimate  'PME1 
iscuerlayed  Management  reserve  mav 
be  established  bv  capturing  the1  cl 1 1 
terence  between  the  bid  ot  the  signed 
contract  and  the  PME  point  estimate 
or  budget  tor  t he  c  ontrac  t 

Strength  of  Proposed  Method 

Alter  developing  and  displacing  t In 

i  os t  estimate  i'p  a  :isk  bus.s  i.  above 
the  SSEB  then  has  the  basis  .  m,  a  sure 
three  aspects  ot  cost  realism  a  the 
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probability  or  risk  of  the  contractor's  cost  realism  of  the  proposal);  c)  the 
bid  being  exceeds  1;  b)  the  amount  of  value  of  standard  deviation  which  pro- 
over  or  under  bid  relative  to  SSEB  vides  the  risk  dollar  exposure  in  con- 
mean  'it  is  recommended  that  this  tractor's  bid.  The  general  interpreta- 
point  be  the  reference  for  measuring  tion  is  that  the  further  the  contractor's 
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bid  is  from  the  SSEB  mean  tor  that 
proposal,  the  more  unrealistic  the  con¬ 
tractor's  bid.  The  other  measure  is  the 
amount  of  program  cost  risk  dollars  as 
indicated  by  the  standard  deviation. 
The  smaller  the  standard  deviation  the 
less  dollar  overrun  exposure  one  has. 
Another  benefit  to  this  methodology 
is  that  now  the  proposed  contract  cost 
can  be  compared  with  the  program 
manager's  budget  which  would  show 
how  much  of  the  contract  risk  is 
budgeted.  (Note  that  before  com¬ 
parison,  items  in  the  program 
manager's  budget  not  applicable  to  the 
contract  award  should  be  subtracted. 
Items  like  budget  for  future  known  and 
unknown  contract  modifications  and 
budget  for  contract  cost  impacts  due 
to  future  funding  profile  changes  are 
examples  as  well  as  budget  for  other 
government  agencies,  other  support 
contracts).  Requiring  contractors  to 
propose  on  a  risk  estimate  provides  an 
additional  two  benefits.  It  forces  the 
contractor  to  bid  more  realistically  and 
it  provides  a  better  basis  for  the  SSEB 
cost  estimate  being  more  accurate  and, 
thus,  a  better  standard  against  which 
to  measure  cost  realism. 

Summary 

It  the  objective  of  emphasizing  cost 
realism  in  the  SSEB  process  is  the 
ultimate  objective  of  awarding  a  cost 
type  contract  at  a  reasonable  target 
cost  and  with  reasonable  expectation 
of  minimal  overrun,  then  a  change  is 
required  in  the  request  for  proposal 
and  SSEB  process.  This  change  re¬ 
quires  the  SSEB  cost  reasonableness 
measurement  process  to  be  done  on  a 
cost-risk  basis  and  to  improve  the 
overall  process  requires  contractors  to 
propose  on  a  cost-risk  basis  with 
backup  rationale  which  supports 
reasonableness  and  credibility  ot  the 
cost  proposal.  This  rationale  must  ad¬ 
dress  credibility  ot  the  proposed  ap¬ 
proach  and  schedule  tor  satisfying  the 
request  tor  proposal,  and  must  be  ap¬ 
plied  to  all  major  subcontractors  pro¬ 
posed  by  the  Prime  as  a  a  part  id  the 
Primes  proposal.  The  major  subcon¬ 
tractors  should  be  required  to  bid  to 
the  Prime  on  a  risk  basis  and  these  pro¬ 
posals  submitted  to  the  SSEB  bv  the 
Prime  as  supporting  data  ■ 
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Defense  systems 
acquisition 
experience? 
Teaching  ability? 
Research  ability? 


Understanding  of 
fundamental  DOD 
manufacturing/ 
production 
principles 
related  to  systems 
engineering  and 
DOD  life-cylce 
management 
Knowledge  of 
current  DOD 
production/ 
manufacturing 
policy  practices, 
and  procedures 


The  Defense  Systems 
Management  College 
has  immediate  openings 
for  a  Professor  of 
Manufacturing  Management 


Experience  in 
production  and 
process  planning, 
quality, 
productivity, 
producibility 
engineering  and 
planning,  process 
control,  and 
production 
readiness 


Help  middle 
managers  from 
civilian  and  military 
segments  of  DOD 
improve  their 
effectiveness  in 
defense  systems 
acquisition 


Teach,  consult  and 
research 


Salary  range  is 
$45,763  -  $59,488 
Grade  is  GS-14 
excepted  civil 
service 
Requires  a 
baccalaureate 
degree  in 
engineering  or 
engineering 
management  and 
3  or  more  years  of 
professional 
experience 
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Call  Debbie 
Johnson 

(703)  664-3118  or 
AV  354-3118 
Or  send  a  resume 
or  SF-171 
Personal 
Qualification 
Form  to: 

Defense  Systems 
Management 
College 

ATTN:  DCOS-CP 
Bldg  202 
Fort  Belvoir.  VA 
22060-5426 
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ll  e  business  ot  acquiring  major 
weapons  systems  tor  tht’ 
Department  ot  Petense  is  com¬ 
plex  and  expensive  The  process 
t.ihe-  place  under  intense 
-iruliiH  I",  the  (.  oneress  die  media 
and  (lie  public  l\  lnle  tamili.il  ties  be 
I  ween  tile  I  Vpart  men  I  ot  Pete  n  sc  and 
defense  i  onlrai  tors  mac  exi-t 
ultimatelv  tile  Pepartillent  ot  Petcnsc 
seeks  to  rediu  e  costs  while  c  out  r.U  to!  s 


and  procurement  in  an  attempt  to 
achieve  objectives.  One  aspect  ot  a 
contractors  strategy  is  pricing  the  pat¬ 
tern  ot  prices  charged  tor  units  pro¬ 
cured  over  time.  Do  unit  prices  remain 
lairly  stable?  We  telt  a  contractors 
pricing  strategy  might  be  related  to  his 
financial  condition  and  that  document¬ 
ing  and  discussing  that  relationship 
might  be  ot  value  to  contracting  of¬ 
ficers  and  program  managers  involved 
in  the  acquisition  process. 

Pricing  Strategies 

hirst,  some  background  on  pricing 
strategy.  There  are  numerous  ways  to 
describe  or  categorize  pricing  strategies 
in  general-  but  firms  introducing  new 
products  or  technology  typically  use 
one  of  two  common  product  pricing 
approaches:  penetration  or  skimming. 
'Discussed  by  many  authors, ;  the  two 
strategies  are  widely  understood  and 
used  bv  business  practitioners.  The 
skimming  strategy  calls  tor  high  initial 
prices  followed  bv  lower  prices  at  later 
stages  while  the  penetration  strategy 
calls  for  a  low  initial  price  with  little 
or  no  price  reduction  over  time. 

Ihe  objective  ot  the  skimming 
strategy  is  to  achieve  the  maximum 
profit  in  the  shortest  time  by  charging 
the  highest  price  the  market  will  bear. 
Price  reductions  occur  in  a  series  ot 
steps  timed  to  provide  as  much  profit 
as  possible  .1 1  each  step.  Thus,  the  ad¬ 
vantage  ot  skimming  is  a  more  rapid 
return  on  investment.  Firms  adopting 
a  skimming  strategy  must  keep  one 
step  ahead  ot  competitors:  the  risk  is 
that  competitors  may  underprice  and 
enter  the  market . 

In  contrast  the  objective  of  the 
penetration  strategy  is  to  develop 
wide  product  demand  rapidlv  through 
a  low  initial  price.  Once  the  market 
."as  been  captured  the  firm  can  take 
ad  vantage  ol  eit  her  price  i  nc  leases  or 
i  os',  reductions  to  earn  additional 
profits.  I  lie  firm  s  established  market 
position  dampens  incentives  ol  <  om 
pel  i1  "is  :  c  >  enter  I  he  mark  et 

face’  discussed  the  interest  ol 
•'•I'.'efs  and  sellers  legardmg  pricing 
m'c  ,'\  I  here  ale  c  liar  me  entices  tor 
a  '  I  lei  :  o  c  one  eal  pi  u  ing  s|  ralegv  ll 
i  ■ 1  m  pi  '  it  o>  -  detect  a  skimming 
'  •  i‘e.:\  I  In",  mu',  c  ounter  c\  ith  a 

-  ■  '  p:  "  '  i: ;• :  i  apt i:r c-  a  market 

. : 1  . :  ;t  oi  s  dele  i  t  a  pellet  ra 

■  :  i  a'i  .  :  rev  will  antic  ipate  !  he 

I  -  a  j'l  u  e  :  e d  a  'ion  m  the  tutufe 


and  be  more  encouraged  to  enter  the 
market.  There  are  economic  benefits 
to  be  gained  by  a  buyer  detecting  pric¬ 
ing  strategy  Customers  detecting  a 
skimming  strategv  can  delay  pur¬ 
chases  to  obtain  a  more  favorable 
price  in  the  future.  An  early  buyer 
ot  a  new  product  would  prefer  to  ac¬ 
quire  from  a  penetrator  while  a  late- 
lilc  buyer  would  likely  prefer  to  ac¬ 
quire  from  a  skimmer. 

Clearly  defense  acquisition,  par¬ 
ticularly  for  major  weapons  systems, 
is  specialized  in  nature. A  The  prod¬ 
ucts  and  the  market  are  not  typical  ot 
products  and  markets  in  general.  Ma¬ 
jor  weapons  systems  incorporate 
significant  innovation  with  state-of- 
the-art  hardware  and  substantial 
uncertainty  in  development.  Products 
involving  significant  innovation  offer 
the  possibility  of  '  learning"  over  time 
and  provide  the  greatest  leeway  in 
choosing  a  pricing  strategy.5  The 
market  for  defense  systems  is  unusual, 
with  a  single  (monopsonistic)  buyer 
and  usually  only  a  few  (oligopolistic) 
sellers. p  Yet,  varying  incentives  exist 
tor  skimming  and  penetration  within 
this  market.  Sellers  that  penetrate  risk 
program  termination  before  long-run 
profits  can  be  realized.  Sellers  that 
skim  risk  program  termination  or  cur¬ 
tailment  clue  to  excessive  price,  and 
risk  competitor  entry  by  encouraging 
the  buyer  to  seek  lower  prices 
elsewhere.  However,  a  large  monop¬ 
sonistic  buyer,  like  the  Department  ot 
Defense,  may  be  in  a  position  to 
thwart  a  skimmer  s  attempt  at  a  high 
initial  price  by  encouraging  competi¬ 
tion  through  second  sourcing.  Detec¬ 
tion  ot  a  sellers  strategy  can  poten¬ 
tially  lead  to  significant  cost  savings 
tor  the  informed  buyer. 

Our  feeling  is  that  pricing  strategy 
may  be  influenced  by  financial  con 
dition.  and  that  pricing  strategy  may 
be  detected  using  financial  ratios 
public  Iv  available  before  product 
int  roduc  t  ion 

Financial 

Condition  and  Pricing 
Strategy 

l-_.lt  h  ot  the  two  strategies  can  be 
described  in  terms  ot  the  relationship 
between  two  variables  the  price  ol  the 
tils!  unit  si. Id  and  the  rate  ot  price 
rc  dui  lion  I'vei  time  'skimmers  exhibit 
a  high  first  unit  pm  i  and  a  steep  pr  ic  c 
1 1 -due  !  I"H  1  II!  \  I  is  h lie  pellet!  at o|  s  i  s 


hibit  a  low  tirst-unit  price  and  a  flat  I 
price-reduction  curve  In  principle  a 
seller  could  be  mditterent  to  the  two 
strategies.  A  high  initial  price  coupled 
with  steep  price  reduction,  or  a  low  in¬ 
itial  price  coupled  with  flatter  price 
reduction  could  result  in  the  same  pres¬ 
ent  value  tor  a  product  and  the  same 
net  economic  benefit.  Neither  strategy 
is  inherently  more  profitable.  In  prac¬ 
tice,  however,  there  are  likely  to  be  in¬ 
ternal  factors  related  to  financial  con¬ 
dition  that  may  result  in  one  or  the 
other  strategy  being  preferred. 

Such  factors  mav  be  reflected  in  the 
firm  s  financial  ratios.  Readers  familiar 
with  accounting  or  financial  statement 
analysis  are  well  aware  that  numerous 
ratios  can  be  calculated  from  financial 
statement  data,  and  that  these  ratios 
can  be  categorized  in  many  ways.  It  is 
not  unreasonable,  however,  to 
categorize  financial  ratios  into  live 
broad  categories  representing  five 
aspects  ot  financial  condition: 

—  Profitability  (return  on  investment  t 
—Short-term  liquidity 

—  Solvency  (capital  structure) 

—  Activity  (turnover) 

—  Capital  investment. 

Our  object  here  is  to  suggest  why  pric¬ 
ing  strategy  may  depend  on  these 
aspects  ot  financial  condition. 

Profitability.  As  we  indicated, 
neither  skimming  nor  penetration  is,  in 
the  long  run,  inherently  more  protit- 
able:  the  central  difference  between  the 
two  is  in  the  timing  ot  protits.  Skim¬ 
ming  provides  lor  high  protit  recogni¬ 
tion  immediately  after  product  in¬ 
troduction:  penetration  otters  the 
possibility  ot  cost  reductions  or  price 
increases  and  higher  profits  at  a  later 
stage.  Since  executives  frequently  are  j 
compensated  on  the  basis  ot  protit 
measures,  one  might  expect  concern 
regarding  the  effect  ot  pricing  strategy 
on  such  profitability  measures  High 
protitabilit v  before  introduction  ot  a  ! 
new  product  may  be  associated  with 
continuing  demand  tor  high-prolit 
projects  in  the  short  run  This  is 
because  low  profit  proiecN  mav  reduce 
the  tirm*-  overall  average  protitabili 
tv.  Penetration  when  compared  to 
skimming  increases  the  probabilitv 
that  average  pi.'titabilitv  measuti  s  will 
decline  alter  piodiut  intmduition 
'sm  h  .1  dei  line  is  mi>ic  likelv  toi  linns 
with  bud'  pi  i't!(abilil\  bet  "re  product 
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introduction  Consequently,  we  expect 
firms  with  h  i^h  protit  ability  to  have  a 
stronger  preference  tor  skimming. 

Short-Term  Liquidity.  Initiation  of 
new  products  may  require  substantial 
outlays  to  finance  inventories,  produc¬ 
tion  volume  and  product-introduction 
eo  ts.  Skimming,  because  ot  taster 
payback  due  to  higher  initial  prices,  is 
appropriate  tor  firms  needing  funds  in 
the  short  run.'  Firms  with  a  poor 
short-term  liquidity  position  should 
have  greater  difficulty,  or  a  higher 
<  os*  ot  raising  funds  externally  and 
max  preter  to  generate  funds  rapidly 
through  the  product.  Consequently, 
we  expec  t  firms  with  poor  liquidit v  to 
have  a  stronger  preference  tor 
s|  miming,. 

Solvency.  Solvent  v  measures  rellec t 
the  amount  and  tvpe  ot  debt  in  the 
firms  capital  structure  and  indicate 
risk  Analogous  to  the  reasoning 
presented  above  under  liquiditv.  firms 
that  are  more  highlv  leveraged  should 
have  a  higher  cost  o|  raising  new 
capital  and  max  exhibit  a  preterem  e 
lot  raising  tunc1  : hrough  t he  new  proi 
eel  bv  s[.  miming  In  addition  alter¬ 
native  pricing  strategies  dilter  with 


respect  to  king-run  risk.  Penetration 
strategy  requires  that  competition  be 
discouraged,  and  that  returns  be 
earned  over  the  long  run  to  be  suc¬ 
cessful.  However,  skimming,  by  front- 
ending  profit,  reduces  the  risk 
associated  with  future  uncertainty  in 
the  products  market.*1  Firms  with 
greater  risk  may  preter  to  reduce  I  lit  ure 
uncertainty.  For  these  reasons,  we  ex¬ 
pect  firms  with  poorer  solvencv 
measures  'more  debt1  to  prefer 
skimming. 

Activity.  If  a  firm  lias  limited  man¬ 
ufacturing  capacity  a  small  volume 
but  highlv  profitable  market  approac  h 
Me.  skimming1  m.iv  be  the  most 
economic  b  Penetration  requires  wide 
dillusion  o|  the  product  to  be  sue 
cesstul  and  c  oiisequenllv  icq.nres 
greater  availahilitc  ot  resources  oi 
cap.nitv  to  support  greater  x  oleum 


Measures  ot  activity  or  turnover  reflect 
the  level  ot  sales  generated  on  assets 
and,  consequently,  indicate  the  degree 
to  which  resources  or  capacity  are  be¬ 
ing  utilized.  One  would  expect  that 
firms  fully  using  existing  capacity  may 
be  constrained  from  following  the 
higher-volume  penetration  strategy, 
while  firms  not  fully  using  existing 
capacity  should  preter  penetration  to 
increase  the  probability  that  their 
capacity  is  put  in  service.  Consequent¬ 
ly.  vve  expect  firms  with  low-activity 

ratios  to  preter  penetration. 

i 

Capital  Investment.  Somewhat 
i  analogously,  investment  in  new  assets 
may  indicate  future  pricing  strategy. 

!  Skimmers  should  have  less  need  to  ex¬ 
pand  capacity,  while  penetrators.  ex¬ 
pecting  tti  generate  volume  through 
low  initial  price,  have  a  greater  need 
to  expand.  Major  investment  in  capaci¬ 
ty  could  signal  a  penetration  strategy 
and.  given  an  increase  in  new  assets 
penetration  would  be  preferable 
strategy  to  assure  utilization  ot  those 
assets.  Consequently  we  expect  firms 
with  high  ratios  ot  new  investment  in 
;  plant  and  equipment  relative  to  ex 
!  isting  assets  or  levels  ot  aetivitx  to 
!  preter  penet ration . 

To  Summarize.  We  expet  t  firms  that 
skim  as  lompared  to  turns  that 
penetrate  to  exhibit  measures  ot  high 
prolitahilitx  poor  liquiditv  pool 
solvent  \  high  asset  utilization  and 
low  inxestment  m  tapat  itx  betoie  new 
produi  t  intrndui  lion  in  the  follow  mg 
so.  t  ions  XX  e  pro x  ide  ex  ;de!1(  e  I  ’  om  .111 
.m.iix  s|-  ot  .1  s.impk  oi  i  'OP  .  oil ; :  .1. 
tom  that  some  ot  thest  expet  ltd  rei.i 
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Slope  of  Price  Reduction  Curve 

As  indicated,  the  two  strategies  can 
be  described  in  terms  ot  the  relation¬ 
ship  between  first  unit  price  and  the 
subsequent  price-reduction  curve. 
Learning  curves  can  be  used  to 
distinguish  the  two  strategies.  Learn¬ 
ing  curve  theory10  describes  the 
decline  in  per  unit  production  costs  a 
manufacturer  experiences  with  increas¬ 
ing  volume.  The  learning-curve  con¬ 
cept  originated  from  the  observation 
that  individuals  performing  repetitive 
tasks  tend  to  exhibit  a  rate  ot  improve¬ 
ment,  but  there  are  many  reasons  tor 
reduction  of  costs  over  repetitive 
operations:  more  efficient  labor,  less 
material  from  reduced  scrap  and 
waste  and  higher  productivity  from 
improved  processes.  Thus  a  learning 
i  urve  more  generally  can  be  referred 
to  as  a  cost -reduction  curve.  A  per-unit 
reduction  can  be  extended  conceptual¬ 
ly  to  the  measure  ot  price  per  unit. 
Thus  learning  curves  can  be  used  to 
represent  price-reduction  curves. 

The  learning-curve  function  relates 
a  dependent  variable  'price1  with  an  in¬ 
dependent  variable  'volume'  as 
follows 


or  in  le'g  form 

In  I’  =  1  n  A  + 


B  '  1  n  X  i 


Where  I’  is  the  price  of  the  Xth  unit 
produced  and  A  is  the  price  ot  the  first 
unit  If  prices  are  level  as  volume  'X' 
increases  then  the  exponent  B  is  zero 
B  is  negative  when  prices  decline  with 
volume.  The  slope  ot  the  learning 
curve  S  is  related  to  B  as  follows: 


A  slope  of  1 . 00  implies  a  horizontal 
line  i.e  no  pru  e  reduction  I  he  lower 
the  dt  i  im.il  value  ot  the  slope  the 
higher  the  pric  e  red  in  t  ion  rate  For  ex¬ 
ample  gOO  is  a  steeper  'taster  price 
redsa  ! ion  i ate  than  °00 

In  our  stuck  we  used  slopes  ot 
learning  curves  tit  to  actual  prices  to 
reflect  pru  mg  stra'egc  Kelativelv  high 
c  allies  tor  *s  arc  consistent  with 
penetration  -flat  slope-  while  1  c  •  •  ■  r 
c  a!:,e-  are  consistent  with  skimming 

steeper  rediic  lion 

We  examined  price  reduc  ten  c  urves 
tor  ma|or  militarv  weapons  sv'lrms 

toyon >i  M, ictagc'c 


F-860 

North  American 

51 

.926 

F-89D 

Northrup 

51 

.885 

F-86F 

North  American 

51 

.870 

F-84F 

Republic 

51 

.725 

F-100A/C 

North  American 

52 

.839 

F-1B/C/MF-1C 

North  American 

52 

.783 

F-102-A 

General  Dynamics 

53 

.724 

F-101-A/B/C 

McDonnell  Douglas 

54 

.802 

F-100D 

North  American 

54 

.934 

A-4B 

McDonnell  Douglas 

55 

834 

B-52G 

Boeing 

57 

.869 

F-106A/B 

General  Dynamics 

57 

.837 

A-4C 

McDonnell  Douglas 

57 

.894 

F-105B/D 

Republic 

57 

.759 

F-4A/B 

McDonnell  Douglas 

59 

.834 

P-3A 

Lockheed 

60 

.718 

A-6A 

Gruman 

61 

.829 

RIM-24B 

General  Dynamics 

61 

.923 

A-4E 

McDonnell  Douglas 

61 

.892 

RIM-2E 

General  Dynamics 

61 

.930 

F-40 

McDonnell  Douglas 

64 

.886 

A-7A/B 

Vought 

65 

.852 

P-3B 

Lockheed 

65 

.910 

RIM-66A 

General  Dynamics 

66 

.763 

RIM-67A 

General  Dynamics 

66 

.825 

AIM-7F 

Raytheon 

68 

.773 

A-7D 

Vought 

68 

.950 

S-3A 

Lockheed 

72 

.846 

F-15A 

McDonnell  Douglas 

73 

.917 

AGM-78D 

General  Dynamics 

73 

1.088 

AH-1S 

Bell 

75 

.891 

AH-1T 

Bell 

76 

1.021 

F/A-18A 

McDonnell  Douglas 

79 

.860 

AIM-7M 

Raytheon 

80 

.880 

BGM-109 

General  Dynamics 

80 

.943 

'aircraft  and  missiles)  acquired  by  the 
Department  of  Defense  from 
I051 -logo  Two  publications,  US. 

'  Military  Aircraft  Cost  Handbook  and 
U  S  , Military  Missile  Cost  Hand- 
\  book  11  provide  a  wealth  of  data  on 
per-unit  costs,  volume  and  cost  pat- 
,  terns  for  most  major  U  S.  aircraft  or 
missile  systems.  Price-reduction  slopes 
using  learning  curves  (constant  dollars) 
are  inc  ludcd 


Handbooks  provide  data  tor 
numerous  weapon  svstem  programs 
whu  h  had  to  pass  three  (liters  to  be  in 
eluded  in  the  study  hirst  programs 
kid  to  run  at  least  1  vc.irs  in  order  to 
1  ale  1. late  mcamnglul  slopes  Sec  ond 

pi  ogi  arris  where  learning  c  urves  tit  to 

the  raw  pm  1  data  provided  a  poor 


"fit"  were  elinu.  '.ed.  Since  the  pur¬ 
pose  here  is  to  explain  variations  in 
price-reduction  curves,  only  programs 
with  well-defined  price-reducion  slopes 
were  included.  An  R:  value  in  excess 
of  .6  was  used  as  a  cutoff  for  program 
inclusion.  Third,  financial  statement 
data  for  the  year  before  program  in¬ 
itiation  had  to  be  available  without 
unreasonable  search. 

The  remaining  group  consisted  ot  3? 
programs.  Project  identifiers,  the 
manufacturer,  the  year  of  project  in 
itiation  and  price-reduction  slopes  tor 
the  35  programs  are  provided  in  Table 
1  Slopes  around  .800  to  DOOurecom 
men  tor  complex  high -te<  hnologv 
pioducts  although  more  e'tfeme 
value--  are  not  rare  -e-  the  -ample  tinr- 
seelll  to  he  representative  ot  the  prod 
net  t  \  pe 
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1.  Return  on  Assets 

2  Return  on  Equity 

3  Return  on  Capital 
4.  Profit  Margin 

5  Gross  Margin 


Net  Income/Total  Assets 

Net  Income/Stockhoiders  Equity 

Net  Income/Non-Curr.  Liab  +  Stockholders  Equity 

Net  Income/Sales 

Gross  Margin/Sales 


6  Current  Ratio 

Curr.  Assets/Curr.  Liab. 

+ 

.25  • 

7  Quick  Ratio 

(Cash  +  Mkt.  Sec.  +  Acct.  Rec.)/Curr.  Liab. 

+ 

-  .03 

8  Current  Asset  Ratio 

Curr.  Assets/Total  Assets 

+ 

-.16 

9.  Working  Capital  Ratio 

(Curr.  Assets-Curr.  Liab. (/Total  Assets 

+ 

.12 

10  Receivables  Turnover 

Sales/Accounts  Receivable 

+ 

.17 

11.  Debt  Ratio 

Total  Liab  /Total  Assets 

-.08 

12  Equity  to  Debt 

Stockholders  Equity/Total  Liabilities 

4- 

.11 

13  Curr  Debt  Ratio 

Curr  Liab. /Total  Assets 

- 

-  .29*  • 

14  Non-Curr  Debt  Ratio 

Non-Curr.  Liab/Total  Assets 

- 

.18 

15  Interest  Coverage 

Operating  Income/Interest  Expense 

4 

-.04 

16  Asset  Turnover 

Sales/Total  Assets 

_ 

-  23. 

17  Plant  Asset  Turnover 

Sales/Plant  Equipment 

— 

-.03 

18  Inventory  Turnover 

Cost  of  Goods  Sold/Inventory 

- 

-  05 

19  Working  Capital  Turnover 

Sales/(Curr  Assets-Curr.  Liab) 

— 

-  19 

20  Investment  to  Sates 

21  Invest  to  Funds 

22  Investment  to  Assets 

23  Investment  to  Plant 


Investment/Sales 
lnvestment/(Net  Income  *  De 
Investment/Total  Assets 
Investment/Plant  &  Equipment 


Depreciation) 


.25 

.41  •  • 
08 
01 


Significant  at  <«  10 
►  Significant  at  -  05 


Financial  Ratios  and 
C  orri’lafion  Analysis 
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huch  ratio  u,i*  correlated  with  price 
reduction  slopes.  I  spec  ted  si^ns 
assuminn  ratios  are  related  t c *  pricing 
•4 rateyv  in  the  way  we  antic  ipated 1  and 
*  orrel.it ion--  are  reported  in  the  nyht 
hand  c olumns  ol  Table  2  Several  tind- 
are  interesting  In  general  correla¬ 
tions  are  not  eMremclv  hiyh  and  are 
not  seanil  ic  ant  at  traditional  level*  • 
However  one  ratio  within  each 
.  iti  .tore  w  sianitlc  ant  and  tile  mkiw  ot 
the  i  orrel.i’iiitis  tot  t|io*e  ratios  arc 
i  on  sistrn!  w  it  I  w  hat  w  *  expected  The 
t,n  '  that  there  w  one  ratio  in  each 
i  a'eitor'.  that  is  sianitu  antic  assoc  ia!ed 


related  In  spite  ot  the  insignificance  ot 
the  other  ratios  all  except  live  have 
siyns  as  predic  ted.  All  ratios  within  the 
activitv  and  investment  categories 
have  the  expected  * lyn  All  five  ratios 
with  unexpected  siyn*.  have  low  cor- 
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ratio  at  a  time  to  a  model  depending 
on  which  ratio  most  assists  in  explain¬ 
ing  the  variable  ot  interest  in  this  ease 
price-reduction  slope  Bv  selectively  m- 
lluenc  ing  the  entrv  ot  variables  into  the 
model  during  the  stepwise  procedure 
a  researcher  has  some  control  ot  the 
model  that  results 

We  investigated  various  model'  in 
a  heuristic  and  iterative  fashion.  We 
were  concerned  with  three  qualitative 
tailors  in  constructing  the  model 

I'arsimonv  Ur  pretence!  a  mode! 
with  tew  ratios 

I  uc  h  ot  interrelationship  be! wee  n 
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We  considered  three  statistical  items 
to  determine  when  we  had  arrived  at 
a  good  model: 

The  overall  significance  ot  the 
model  i  h  value 1 . 

The  significance  ot  individual 
r.i tic"  in  the  model  t  slat wt u s  tor  ratio 
c  1  >et I ic  ients 

I  he  explanatory  power  ot  the 
model  aduisted  Ix-squarcd  values 

I  able  '  p:o\  ides  detail  on  out  best 
modi  I  I  1 'of  in.it  a!  tlu  table  several 
:  1 1  t '  '■  s  ,  1 1 1  • '  t  : '  1 ' '  >  1  I  u  model  ;>  signitl 
1  a  at  1 i  tall  and  lias  a  1 1  asonablv  high 
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What  Doe*.  Model  Tell  L' s ? 


In  general  tile  model  demonstrates 
that  pricing  strategy  is  significantly 
associated  with  financial  condition 
and  suggests  that  financial  ratios 
.nailable  before  manufacture  and 
delivt  tv  ot  a  new  weapons  system  mav 
be  uset ul  tor  detec  ting  contractor  pric¬ 
ing  strategy.  A  relatively  small  collec¬ 
tion  ot  ratios  appears  to  explain  a  fair 
amount  ol  the  variation  in  price- 
reduction  rates. 

I  he  individual  ratios  in  the  model 
are  interesting.  As  a  group  they  tend 
to  involve  measures  ot  current  assets 
and  current  liabilities  The  Current 
ratio  compares  liquid  assets  with  near¬ 
maturing  liabilities.  The  Receivable 
Turnover  ratio  reflects  how 
rapidlv  accounts 
are  collected, 
both  are  tradi¬ 


tional  indicators  ol  liquidity  Solven¬ 
cy  measures  are  designed  to  reflect  the 
degree  ot  debt  in  a  firm's  capital  struc¬ 
ture.  The  solvency  ratio  that  proved 
most  important  here,  the  Current  Debt 
ratio,  emphasizes  the  firms  current 
liabilities  only.  The  activity  measure  in 
the  model,  Inventory  Turnover,  relates 
sales  volume  to  a  current  asset- 
inventory.  Even  the  Investment  of 
Funds  ratio  while  designed  to  indicate 
the  level  of  new  investment  in  long-run 
capacity,  may  capture  current  aspects 
of  operations;  a  high  Investment  to 
Funds  ratio  can  mean  a  firm  is 
generating  relatively  little  cash  flow 
from  current  operations  relative  to  the 
amount  of  funds  invested  in  new- 
capacity.  In  short,  each  ratio  can  be 
seen  as  reflecting  some  current,  rather 
than  long-term, 
aspect  of 
financial 
condition. 


SLOPE  =  .7745  +  .0469  (R1)  +  .0075  (R2)  -  .3042  (R3)  +  .0007  (R4)  -  .0051  (R5)  +  .1350  (R6) 


R1 

CURRENT  RATIO 

LIQUIDITY 

1.08 

15 

R2 

+ 

RECEIVABLE  TURNOVER 

LIQUIDITY 

1.71 

05 

R3 

- 

CURRENT  DEBT  RATIO 

SOLVENCY 

1.28 

11 

R4 

+ 

INTEREST  COVERAGE 

SOLVENCY 

1.98 

03 

R5 

- 

INVENTORY  TURNOVER 

ACTIVITY 

1.11 

14 

R6 

INVESTMENT  TO  FUNDS 

INVESTMENT 

3.56 

001 

F  Value 

5.29 

Significance  Level: 

004 

R? 

.665 

Adjusted  R? 

.539 

strategy,  nrm..  that  had  significant  ' 

new  investment  in  assets  or  were  3 

poorly  utilising  existing  assets  before  Sel 

product  introduction  tender!  to  prefer  ,\li, 

penetration.  o-l 

We  dir!  not  find  Profitability  ratios  tK 
to  be  significantly  related  to  pricing 
strategy  after  controlling  tor  ratios  4. 

from  the  other  categories.  Given  the  wh 
nature  ot  the  sample  (government  con-  ing 


‘ailed  Statement  ot  hanges  m  Finan¬ 
cial  Position  data  was  unavailable  tot 
mans  turns  particularly  those  from 
earlier  year*-,  F.ach  ot  the  tour  invest¬ 
ment  ratios  is  an  attempt  to  deflate  in¬ 
vestment  tor  some  aspect  ot  tirm  size. 

14  V\e  used  one-tailed  tests  ot 
'i.anit k unie  because  our  hvpotheses 
spec  itied  a  directional  relationship  bet¬ 
ween  slopes  and  ratios. 

15.  I’nadjusted  K:  always  increases 
as  more  ratios  are  added  to  a  model 
A  model  can  be  tound  that  statistical- 
k  explains  all  variation  in  price- 
reduition  slopes  simply  by  including 
enough  ratios.  Adjusted  R-  is 
deflated  to  allow  tor  the  number  ot 
ratios  in  the  model. 

Ip  l\e  tound  several  candidates  tor 
best  model.  Models  including  onh 
tour  or  live  ratios  actually  had  higher 
F  values  and  adjusted  R:  values 
almost  as  high  as  the  six  ratio  model 
discussed  in  the  paper.  In  addition  all 
ratio  coetticients  in  the  tour  and  live 
ratio  models  were  sjgniticant  at  .  10  or 
better.  However,  the  six  ratio  model 
has  the  highest  adjusted  R-  and  in 
eludes  all  of  the  ratios  that  were  im- 


Ihel  S  Army  Materiel  Command 
AMC  1  Alexandria  Ya.  celebrated  its 
25th  anniversary  August  1  1^87. 

F.stablished  May  8.  I°c2  and  activated 
the  following  August  1  in  a  sweeping 
reorganization  ot  the  Army  AMC 
I'ec  ame  the  first  peacetime  inaior  field 
omii'.ind  ot  the  Department  ot  the 
Arm v  responsible  tor  developing  and 
-upplvmg  materiel  and  logistics  tor  the 
Arm  v 
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['ortant  vat  tables  in  anv  o!  the  other 
candidate  models  It  provides  a  better 
basis  tor  discussion. 

17.  We  conducted  a  factor  analvsis 
on  the  full  set  ot  ratios  and  except  tor 
some  overlap  between  the  current  ratio 
and  the  current  debt  ratio,  each  ot  the 
six  ratios  in  the  model  is  associated 
with  a  distinct  individual  dimension  ot 
financial  condition. 

18.  However,  models  replacing  the 
Current  Debt  ratio  with  the  total  Debt 
ratio  and  the  Inventory  Turnover  ratio 
with  the  Asset  Turnover  ratio  were  still 
in  excess  ot  .40. 

1°.  Defense  Financial  and  Investment 
Review  (lT.S.  Department  ot  Defense. 
June  1°85>. 

20.  See  for  example  Kennedy,  In¬ 
centive  Contracts  and  Cost  Growth. 
Report  No.  BRML-80-5103.  (Air  Force 
Business  Research  Center.  Wright- 
Patterson  AFB.  October  31.  l°83)and 
Kennedy,  I..  The  Appropriate  Use  of 
Incentive  Contracts,"  lu8’5  Pio- 
ceedings  Federal  Acquisition  Sym¬ 
posium.  Defense  System  Management 
College,  Ft.  Belvoir.  Va..  pp.  217-221. 


21.000  military  and  lo°,000  civilians 
whose  mission  was  to  improve  the 
degree  ot  excellence  in  research,  main¬ 
tain  quality  ot  thought  and  creativity, 
and  apply  these  where  needed. 

Today’s  AMC  is  one  ot  the  largest 
and  most  complex  commands  in  the 
Army,  spending  almost  halt  ot  the 
total  ITS.  Army  budget  and  em¬ 
ploying  more  than  0  000  military  and 
1  13  000  t  ivilians  world-wide  Its  mis¬ 
sion  is  to  support  the  soldier  in  the  held 
bv  managing  weapon  systems  and 
equipment  Irom  the  initial  concept 
development  and  fielding  through  its 
I'lte  c  vc  le  to  final  disposal 

During  the  vent  the  38°th  Amu 
Hand  AMA  s  Own 1  will  tour  the  coun 
trv  celebrating  this  silver  anniversary 
and  the  Bicente  nnial  ot  the  ISA  on 
stitution  An  anmviTsirc  exhibit  will 
be  shown  m  Octobei  at  flu  .m.m.a 
Assoc  lation  ot  tlie  I  nited  states  Amu 
i  orin  ntiai’  Washing.ton  1 1  A  • 


An  interesting  Procurement  Round 
Table  report  has  been  prepared  by  one 
ot  its  directors,  Professor  Ralph  C. 
Nash.  Jr. 

The  report  concludes  that  a  totally 
new  proprietary  rights  policy  embody¬ 
ing  five  elements  is  necessary  to  pro¬ 
vide  the  benefits  ot  competition,  while 
protecting  the  rights  of  contractors  and 
greatly  simplifying  present  regulations 
and  contract  clauses.  The  elements 
follow. 

—  Establish  a  single  regulation  for  use 
by  the  Department  of  Defense  and  the 
civilian  agencies. 

—  Separate  the  proprietary  rights 
policies  covering  technical  data  from 
computer  programs. 

—  Permit  contractors  to  retain  com¬ 
mercial  rights  in  innovative  work  done 
on  government  contracts.  At  the  same 
time,  this  policy  must  (1)  permit  the 
government  to  use  all  technical  data 
developed  on  government  contracts 
tor  internal  purposes,  and  (2>  require 
the  contractor  to  license  companies  to 
use  the  data  in  government  competi¬ 
tions. 

—  Pay  contractors  a  royalty  when  they 
license  competitors  as  compensation 
for  the  successful  completion  ot  a 
developmental  effort. 

—  Avoid  violating  the  proprietary 
rights  ot  contractors  by  controlling  the 
techniques  used  to  obtain  competi¬ 
tion.® 


The  Troop  Support  A  ornmand  s 
Belvoir  KDttK  A  enter  has  awarded  ,i 
S2  185  CHX*  prodm  lion  i  ontract  tor  the 
Ainu  s  new  A  tented  lane  Marking 
Mstem  A  I  AMS1  which  will  help 
whale  drivers  lollow  s.ile  paths 
i  leared  through  minefields  Mounted 
on  the  real  ot  the  lead  minefield 
breaching  vehule  A  1  AM  dispenses 
markers  tittevl  w  ith  i  olored  (lags  that 
,  ...  d;.:  .'  .  'hi  d.l\ 
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Lieutenant  Colonel  Robert  M.  Baker.  USA 


uring  the  past  3  years  the 
Army  undertook  a  major  ef¬ 
fort  to  streamline  its  materiel 
acquisition  system  and  pro¬ 
cedures  for  developing  and 
documenting  materiel  requirements.  In 
this  paper,  I  examine  changes  in  the 
materiel  requirements  documentation 
system,  and  conclude  these  changes 
have  a  positive  impact.  Since  problem 
areas  remain,  recommendations  are 
made  tor  further  improvements. 

One  of  the  Department  of  Defense 
Acquisition  Program  Initiatives 
•  Carlucci  Initiatives)  directed  defense 
agencies  to  reduce  the  number  of 
DOD  directives  and  eliminate  our  non¬ 
cost  effective  contract  requirements.'  1 
This  initiative  was  the  Genesis  ot  the 
Acquisition  Streamlining  Program;  the 
Arms’  has  responded  and  totally  revis¬ 
ed  material  acquisition  policies  These 
major  <  hanges  were  published  in  Army 
Regulation  70-1  System  Acquisition 
Policy  and  Procedures  dated  Nov  12, 
l°3o  This  regulation  makes  tailoring 
ot  the  tradition.il  acquisition  life-cycle 
model  a  standard  procedure  and  in- 
•loduces  the  Army  Streamlined  Ac 
quisition  Process  'ASAP1  the  primary 
torm  ■  ’ !  streamlining  tor  low-risk 
developments  In  conjunction  with 


these  efforts.  General 
Richard  H.  Thompson, 
who  was  Commander 
of  the  U.S.  Army 
Materiel  Command, 
established  "a  goal  for  all 
developmental  programs  not 
to  exceed  four  years--  that  is 
four  years  from  a  Milestone  1 
program  go-ahead  decision  (with 
funds  in  p!ace)to  start  of  production," 
He  said  the  "four  year  developmental 
goal  is  the  heart  of  a  larger  overall  effort 
to  reduce  the  entire  systems  acquisition 
process  from  the  traditional  11-15 
years  to  7-9  years."1 


Therefore,  considerable  attention 
has  been  given  to  requirements 
documents  supporting  the  materiel  ac¬ 
quisition  process.  These  documents  are 
the  foundation  tor  any  materiel 
development:  without  an  appropriate 
requirements  document  a  developmen¬ 
tal  ettort  cannot  begin.  Quality  and 
timeliness  ot  requirements  documents 
are  vital  to  any  ettort  to  shorten  the 
acquisition  process.  In  the  past,  the 
cumbersome  system  tor  developing 
and  staffing  requirements  documents 
often  resulted  in  documents  which  did 
not  adequately  desc  ribe  the  requested 
development  or  included  requirements 


P1  Mu’.Mv 


S  V 


which  were  in  a  nice-to-have  category 
or  demanded  high-risk  technology. 
Without  a  reduction  in  the  processing 
and  approval  time  for  requirements 
documents,  it  is  unlikely  that  the  ac¬ 
quisition  cycle's  length  can  be  reduced 
by  4  years. 

In  addition,  there  are  imperatives 
which  make  increasing  efficiency  of  the 
requirements  process  very  important. 

First,  as  the  required  time  increases 
tor  the  document  to  be  written  and  ap¬ 
proved.  technology  may  change 
thereby  increasing  the  chance  of 
technological  obsolescence  ot  the 
development,  and  decreasing  the  pro- 
bability  ot  defeating  the  threat. 

Second.  the  process  length 
discourages  industry  and  decreases  its 
commitment  to  direct  independent 
research  and  development 
funds  to  the  project. 

■  Lieutenant  Coinin’ I  Bilker 
!■■■*: \netl  to  !>:e  ( of  the  Direetoi 
o  [V'v'i in:  ami  Leuliui lion 

o*  the  Sfi letut  u  i  ■'  nrfciisi'  7  his 
1 1  iii’il;  ’iToff  ret’re~eut'  hi-  eiete- 
lied  t/.v  i:  t  iiiVi'i.i inhi  retire!  the 
■  'I’.’ij;  ‘>t  o'  o.'lii'i  - 


Third,  the  lagging  process  of  docu¬ 
ment  approval  may  make  the  require¬ 
ment  more  difficult  to  defend  in  secur¬ 
ing  funds  from  the  Office  of  the 
Secretary  of  Defense  and  the  Congress. 

Fourth,  the  needed  system  is  un¬ 
necessarily  delayed. 

Finally,  there  is  a  great  waste  of 
manpower  resources  in  the  staffing  and 
restaff  ing  process  which  could  be  more 
gainfully  employed.'’ These  tacts  and 
the  need  to  meet  time  goals  of 
streamlined  acquisition  dictated  an 
overhaul  of  the  Army  materiel  require¬ 
ments  process. 

My  purpose  here  is  to  look  at  the 
Army  materiel  requirements  process  as 
it  existed  until  early  1986.  to  examine 
reasons  tor  changing  the  process,  to 
look  at  changes  adopted  and,  finally, 
to  assess  progress  in  implementing 
changes  to  the  system.  The  changes 
adopted  are  more  than  cosmetic;  they 
are  substantive  and,  when  fully  im¬ 
plemented,  should  have  a  major  im¬ 
pact  on  the  research  and  development 
and  procurement  portions  ot  the  Ar¬ 
my  budget,  and  on  the  quality  ot 
equipment  provided  to  l  .S.  soldiers 
tor  the  next  20  wars. 
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Because  ot  t ht‘  broad  scope  ot  the  re¬ 
quirements  process  this  paper  con- 
icrns  primarily  the  materiel  re¬ 
quirements  documents  anil  stalling 
and  approval  procedures  tor  these 
documents  in  these  areas  oualityand 
timeliness  ot  the  documents  are  most 
abetted.  To  present  a  manageable 
topic  herein,  training  device  re¬ 
quirements  documents  will  not  be  con- 
|  sidered.  Because  changes  in  the  re- 
:  quirements  process  are  tar-reaching,  it 
i  is  important  that  new  procedures  be 
assessed  early  in  the  implementation  so 
that  necessary  changes  can  be  made 
before  pro’  iem  areas  develop. 

System  Before  Revision 

The  Army  materiel  acquisition  pro- 
1  cess  is  structured  and  designed  to  coor¬ 
dinate  efforts  of  agencies  to  field  an 
operational  capability  in  the  shortest 
time.  Responsibilities  are  generally 
split  between  the  combat  developer 
and  the  materiel  developer.  The  com¬ 
bat  developer,  usually  the  US.  Army 
Training  and  Doctrine  Command 
i  TRADOC*.  is  responsible  for  for¬ 
mulation  of  doctrine  concepts, 
organizations,  materiel  requirements 
and  objectives,*' It  represents  the  user 
community  in  the  materiel  acquisition 
process  The  materiel  developer,  usual¬ 
ly  the  I’.S.  Army  Materiel  Command, 
is  responsible  tor  research  develop¬ 
ment  and  production  ot  a  svstem  in 
response  to  approved  requirements, ' 
We  see  by  these  definitions  that  the 
ma|or  player  in  the  development  ot  re¬ 
quirements  is  TRADOC  but.  in  reali¬ 
se  establishment  ot  a  requirement  is 
,i  [omt  ettort  ot  TRADOC  and  AMC 
from  concept  initiation  through  ap¬ 
proval  ot  the  last  change,  to  the  last 
requirements  doc  umcnt  tor  a  svstem. 


The  basis  tor  development  ot 
materiel  requirements  is  the  concept 
based  requirements  system  iCBRS* 
whic  h  is  the  proc  ess  bv  which  doctrine 
tr, lining  organization  and  materiel 
detic  icnc  ies  are  identified  and  corrected 
through  the  analysis  development  and 
application  ot  operational  concepts 
I  he  (  MRS  responsibility’ ot  the  com 
bat  developer,  starts  bv  developing  an 
umbrella  concept  ot  how  the  Armv 


tight  in  the  future  bv  analyzing 
■  tops  and  requirements  in  context 
i  protet  ted  threats  and  tec  hnologi 
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Armor  School)  develop  operational 
c oncepts  describing  how  each  service 
branch  will  perform  its  mission  within 
that  concept.  These  functional  con¬ 
cepts  torm  the  basis  tor  each  branch 
mission  area  analysis  (\1AA)  which  is 
an  analytical  ettort  describing  the’ 
capability  ot  that  branch  to  execute  its 
approved  concept  as  a  member  ot  the 
combined  arms  team  within  the  um¬ 
brella  concept .  From  this  analytical  et¬ 
tort  deficiencies  are  identified  and  are 


analyzed  to  determine  it  they  can  be 
corrected  by  developing  new  doctrine, 
training  organizations,  materiel  or 
any  combination  thereof.  Solutions 
lavor  doctrine,  then  training  then 
organizations  and,  finally  materiel 
development.  This  decision  hierarchy 
ensures  the  most  economical  solution. 

Alter  each  proponent  establishes 
and  prioritizes  detic ienc  ies  they  are  tor- 
warded  to  TRADOC  to  be  merged  in¬ 
to  an  Armv-wide  prioritized  list  ot 
del u  ienc  ies,  the  battlefield  develop¬ 
ment  plan  1  BDP1.  The  BDI’.  capstone 
document  toi  the  MAA  process,  pro 
vi  des  the  basis  to  toe  us  sc  lied  tile  and 
integrate  1  RADCX  ellorts  in  suppor  t 
ol  i  urrent  and  tut  lire  Army  missions 
Bel  ore  early  l°8o  these  prioritized 
detic  ienc  ies  were  one  ot  the  primary 
generators  ot  science  and  technology 
obieitivcs  which  until  March  Obo 
r:"'  d<  d  p'.o:  ”.  at’.:  "...  ’  .’c  to: 
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As  mentioned  a  materiel  develop 
merit  is  the  least-lavored  solution  to  an 
MAA  deficiency,  been  it  a  materiel 
solution  is  chosen,  a  product  improve 
merit  to  an  existing  system  or  acquisi¬ 
tion  ot  an  ot  I -t  hc-shcl  t  non- 
developmental  item  i\Dl  must  be 
considered  before  deciding  on  a  new 
development  program. 

Using  the  materiel  requirements  pro¬ 
cess,  which  existed  until  late  March 
l°8o,  the  decision  to  initiate  a  new 
system  started  a  mind-boggling 
documentation  process.  First,  tor  all 
systems,  an  operational  and  organiza¬ 
tional  (O&O)  plan  was  required  and 
was  the  program  initiation  document 
used  to  state  the  purpose  ot  the  system: 
where,  how  and  by  what  organization 
it  would  be  used:  and  how  it  would  lit 
into  the  overall  force.  It  authorized 
fund  expenditures  tor  the  concept 
evaluation  phase  of  the  life-cycle 
system  management  model  (I.CSMM), 
phases  ot  which  are  shown  on  the 
traditional  process  chart  at  Figure  1, 
The  O&O  Plan  was  to  contain  ten  or 
fewer  pages,  was  approved  by  the 
Commander,  TRADOC,  and  was  pro¬ 
vided  to  Headquarters,  Department  of 
the  Army  (HQDA>  for  information. 

If  the  system  was  determined  to  be  j 
DOD  major  (costs  greater  than  S200  1 
million  tor  research,  development,  test 
and  evaluation  or  greater  than  Si  | 
billion  in  procurement),  a  justification  : 
tor  major  system  new  start  (IMSN'S) 
was  required.  The  IMSNS  is  a  three-  j 
page  document  which  must  he  submit-  j 
ted  to  the  Department  ot  Defense  to 
obtain  approval  to  start  a  major 
system:  it  approved  by  the  Secretary 
ot  Defense,  it  is  promulgated  to  the  Ar¬ 
my’  in  the  program  decision  memoran¬ 
dum  i RDM)  issued  by  the  Ottice  ol  the 
Secretary  ot  Defense  in  response  to 
that  vears  Army  program  objective 
memorandum,  1  he  PDM  is  the 
authority  tor  program  initiation  and 
expenditure  ot  funds  lor  the  concept 
exploration  phase  ot  the  hte-cvcle 
system  management  model. 

I  he  Fette  r  ol  Agreement  is  the  re 
qiiiremenls  doc  cm  lent  required  to  sup¬ 
port  the  demonstration  and  validation 
phase  ot  1  (.  SMM  It  was  approved  by 
Ic'int  agreement  ot  the  1  KA1XX  l.  om 
mande!  and  the  AMC  C  ommandci  il 
the  Advanced  Development  r  > 
e  alegorv  ot  resean  h  dry  elopmen!  text 
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JiVuiiit-nl  had  to  be  approved  bv  the 
Arrm  Depute  Chief  ot  Statt  lor 
Operations  and  Plans  There  was  no 
limit  on  pages  but  it  was  to  In-  kept 
to  a  minimum  " 

Ihe  required  operational  capabilitv 
document  supported  the  full-scale 
development  phase  ot  the  I.CSMM. 
was  to  be  tour  panes  in  length  and  ap- 
proved  bv  Headc|uarters  Department 
ot  the  Armv.  The  document  s  six  ma¬ 
jor  annexes  except  tc’r  the  coordina- 
tion  annex  were  removed  before  for¬ 
warding  to  Headquarters.  Department 


Armv  tor  approval. 


The  letter  requirement  was  a  docu¬ 
ment  tor  low-value  items  whose  RDTE 
costs  d;d  not  exceed  Sc  million  per 
'/ear  or  procurement  costs  S12 
million  per  vear:  and  total  costs.  KDTH 
plus  procurement  did  not  exceed  S50 
million  tor  the  5-year  program.  This 
single  document  served  the  demonstra¬ 
tion  and  validation  and  tull-scale 
de  velopment  phases  ot  the  I.CSMM 
tor  low -value  items  The  TRADOC 
Commander  and  the  AMC  Com- 
mander  c.'itld  tomtly  approve  this 
doc  ament  whose  goal  was  tour  pages. 
Moreover  similar  documents  existed 
tor  training  devices  and  joint  re¬ 
quirements  and  spec  lal  doc  umenls  ex¬ 
isted  tor  other  categories  like  the  ex¬ 
perimental  Jth  Motorized  Infantry 
Division  and  Special  Operations 
f  ori  es 


The  many  types  ot  requirements 
doc  uments  had  contusing  and  separate 
procedural  rules  approval  authorities 
and  size  goals  l  Sing  the  old  system 
a  maior  system  required  as  a  minimum 
an  OStO  I ’Ian  a  IN1SNS  an  I. OA  and 
a  KOC  each  with  separate  suppor¬ 
ting  cioc  uments  and  annexes. 
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Armor  Artiliei  v  Signal  I  he  author 
was  usually  an  Army  captain  or 
civilian  ot  equivalent  grade.  It  was  sent 
to  the  loint  Working  C  Iroup  i  IWG  i  on 
a  worldwide  basis  tor  review  before 
the  tirst  IWG  meeting  ot  which 
membership  varied  but  usually  includ¬ 
ed  TRADOC  Headquarters . 
TRADOC  schools  TRADOC  In¬ 
tegrating  Centers  (Combined  Arms, 
Logistics  and  Soldier  Support AMC 
the  AMC  proponent  Major  Subor¬ 
dinate  Command  Headquarters 
Department  ot  the  Army  test  agen¬ 
cies:  and  interested  major  army  com¬ 
mands  (MACOMsi.  After  incorpora¬ 
tion  ot  slatting  comments,  joint  work¬ 
ing  group  and  action  ot  liters  rewrote 
the  document  based  cm  agreements 
and  staffed  the  revised  letter  of  agree¬ 
ment  with  the  Joint  Working  Group. 

Based  on  comments  a  coordinated 
draft  letter  ot  agreement  was  written. 
Ot  ten .  a  second  IWG  meeting  was  re¬ 
quired  to  concur  cm  major  revisions 
the  coordinated  draft  I.OA  was  for¬ 
warded  simultaneous!  v  through 
TRADOC  integrating  centers  to 
TRADOC  MACOMs  and  other  ser¬ 
vices.  and  through  the  proponent  ma¬ 
jor  subordinate  command  to  AMC-- 
all  with  comments  tor  TRADOC 
Headquarters.  There  the  action  olticer 
from  the  Office  ot  the  Deputy  Chief  ol 
Staff  tor  Combat  Developments. 
TRADOC  stalled  the  document  inter¬ 
nally  and  based  cm  external  and  inter¬ 
nal  staffing  prepared  a  final  draft  I.OA 
with  the  IWG  chairman,  which  was 
presented  to  the  TRADOC  re¬ 
quirements  review  committee.  Next 
the  TRADOC  action  olticer  again 
revised  the  document  and  sent  it  to  the 
I RAIXX  commander  tor  approval 
before  forwarding  io  AMC  Head 
quarters  tor  stattmg  and  authentica¬ 
tion  Alter  authentication  bv  the  AMC 
C  ommander  the  I  OA  was  published 
bv  IKADOC  Headquarters  it 
advanced  development  costs  did  not 


i'I  c'i  approved  is  a  tribute  to  the  tenaci¬ 
ty  ot  action  officers  Vet.  it  was  possi¬ 
ble  tor  a  development  with  a  very- 
high-level  interest  to  skip  most  slatt¬ 
ing  steps  and  be  approved  rapidly. 
Documents  produced  in  this  manner 
however  often  required  revision  alter 
approval,  —sometimes  at  a  high  cost. 
Although  time  goals  were  provided  to 
months  tor  an  l.OA(.  it  was  apparent 
thev  could  not  be  met  and  were  not 
tracked.  Goals  became  meaningless. 
Based  on  a  sampling  ot  20  re¬ 
quirements  documents  in  W84. 
TRADOC  estimated  formulation  and 
coordination  ot  a  requirements  docu¬ 
ment  took  20-50  months.0 


After  completion,  documents  often 
were  not  high  quality.  Much  can  be  at¬ 
tributed  to  the  process  length,  inex¬ 
perienced  personnel  as  authors,  and 
the  personnel  turnover  in  TRADOC 
combat  development  positions. 


Efforts  to  Improve  the  System 


It  is  apparent  that  the  Army  materiel 
requirements  documentation  system  is 
complicated  and  time  consuming,  a 
tact  evident  to  people  working  with  the 
system.  No  serious  change  came  until 
ettorts  were  initiated  to  shorten  the 
materiel  acquisition  process  in 
response  to  the  Carlucci  Initiatives. 
After  assuming  command  ot 
TRADOC  General  William  R. 
Richardson  made  revision  ot  the  re¬ 
quirements  process  priority.  He  sensed 
Army  frustration  with  the  system  and 
saw  the  solution  as  a  way  ot  shorten¬ 
ing  the  acquisition  cycle.10  He  began 
an  effort  to  study  the  system  in  the 
spring  ot  1°83  which  resulted  in  a 
study  by  the  Army  Science  Board  in 
December  1°8.V  Concurrently  senior 
officials  jt  AMC  'then  DARCOM' 
began  ettorts  to  shorten  the  acquisition 
cycle  and  recognized  the  impact  re¬ 
quirements  doc  uments  had  on  the 
svstem.  1  he  C  ommander  Genera! 
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Encouragement  was  provided  by 
lames  R.  Ambrose  Undersecretary  of 
the  Army  who  felt  TRADOC  should 
lead  in  developing  the  revisions.1-'  and 
bv  General  Maxwell  R.  Thurman. 
Army  Vice  Chief  ot  Statt.  who  fully 
understood  the  need  tor  reducing  the 
staffing  time  tor  requirements 
documents. :J 


In  September  1°84.  the  Army 
Science  Board  Report  ot  the  Panel  on 
Processing  Requirements  Documents 
Study  was  published.  This  report, 
hereafter  referred  to  as  the  Trainor 
Studs',  written  by  Dr.  Richard  J. 
Trainor,  found  1°  specific  weaknesses 
m  Army  procedures  for  processing  re¬ 
quirements  documents.  Dr.  Trainor 
recommended  resolutions.  The 
Trainor  Study  was  the  first  published 
comprehensive  analysis  of  the  pro¬ 
blems  with  the  requirements  process. 
Although  many  of  the  resolutions  were 
not  adopted  the  study  was  received 
positively  by  the  senior  Army  leader¬ 
ship  and  served  as  a  major  input  to  the 
subsequent  efforts  to  revise  the  pro¬ 
cess,  In  October  1°84.  General 
Richardson  directed  his  staff  to  pro¬ 
pose  c  hanges  to  Army  Regulation  71-°, 
Materiel  Objective?  and  Re- 
increment <  Concurrently.  General 
Thompson  directed  his  staff  to  ac¬ 
complish  a  major  rewrite  ot  Army 
Regulation  70-1  pw~tem<  Acquisition 
Police  and  Procedure < 


,‘ide  the  basis  tor  a  subsequent 


materiel  acquisition  regulations  were 
discussed  and  the  need  for  changes 
determined.  It  was  decided  that  Army 
Regulations  70-1  and  71-0  would  be 
concentrated  on.  with  all  supporting 
regulations  to  be  integrated  with  the 
two  and  published  as  soon  as  possible. 
The  2-week  meeting  established  a  con¬ 
sensus  on  problems  with  the  Army  re¬ 
quirements  system  and  established  the 
basis  for  further  rewrite  of  Army 
Regulation  7  l-A  Specific  problem 
areas  with  the  requirements  process 
noted  by  the  study  group  were  as 
follows:  lc> 


—  Initial  requirements  in  the 
documents  are  too  rigid  and  therefore 
cannot  be  met.  They  need  to  be  writ¬ 
ten  as  broad  bands  of  performance. 

— Requirements  documents  guidance  is 
too  general  and  permits  too  much 
discretionary  variation.  On  the  other 
hand,  the  documents  are  generally  too 
specific  and  are  hardware-oriented 
rather  than  performance-oriented. 

—  Policies  on  writing  and  slatting 
documents  are  too  vague. 

—  Requirements  are  allowed  to  creep 
during  develpment,  making  it  difficult 
for  contractors  and  expensive  tor  the 
government. 

—There  are  too  many  outdated  re- 
quirements  still  active  in  the  system 
which  have  not  been  funded. 


The  TRADOC  Headquarters  and 
AMC  Headquarters  >aw  the  need  to 
ensure  regulations  were  coordinated, 
and  they  communicated  this  concern 
to  Headquarters,  Department  ot  the 
Army,  which  was  the  approval 
authority.  In  February  1°85,  Major 
General  lohn  VV.  VVoodmansee,  Ir . . 
Army  Assistant  Deputy  Chief  of  Statt 
tor  Operations  and  Plans,  Force 
Development,  directed  his  statt  to 
develop  a  concept  to  examine  the  en¬ 
tire  materiel  acquisition  process  to  en¬ 
sure  all  efforts  were  coordinated  and 
that  new  revisions  met  Army  needs. 
Alter  discussions  in  the  spring  ot  1°85 
on  the  proper  forum  it  was  decided  in 
early  lime  that  a  2-3  week  studv  be 
i  onduc  ted  bv  a  panel  ot  colonel-level 
representatives  ot  HQDA  TRADOC 
AMC  and  other  agencies  Results 
■.could  i  oordinate  Armv  efforts  and 


—  There  is  no  capability  to  track 
documents  during  the  staffing  and  ap¬ 
proval  process. 


Duplicative  requiri  men!-,  exist 


—  Approval  authority  is  centralized  at 
a  level  too  high  in  the  Army.  Authori¬ 
ty  needs  to  be  decentralized. 

—  There  is  too  much  betting  on  un¬ 
proven  technology. 

—  There  is  no  quick,  simple  process  to 
document  low-cost,  low-risk 
developments. 

—  The  entire  process  ot  documentation 
is  too  complicated  and  time- 
consuming. 

Many  weaknesses  noted  by  the 
Trainor  Study  were  noted  as  major 
problem  areas  by  the  Army  study 
group.  With  the  foregoing  guidance 
and  full  support  of  senior  Army  leader¬ 
ship.  as  evidenced  by  the  following 
statement  by  General  Thurman  in  the 
fall  of  1°85.  TRADOC  began  its  final 
rewrite  ot  Army  Regulation  71-°. 

In  order  to  work  well  with  in¬ 
dustry  we  must  know  enough  to 
give  them  good  direction,  and  we 
must  be  confident  in  demanding 
that  our  standards  be  met.  We 
must  drive  the  requirements  pro¬ 
cess  in  revolutionary  ways.  Our 
requirements  statements  must  be 
simple  and  complete  and  include 
the  soldier.  This  is  not  impossi¬ 
ble.  :" 


—  There  are  too  many  proponents 
without  an  effective  honest  broker. 


—  Performance  characteristics  are 
often  overstated  resulting  in  gold 
plated  systems. 

—  MACOMs  do  not  respond  during 
coodination  ot  requirements 
documents. 

—  Cost  and  schedule  drivers  are  not 
addressed  diming  document 
development . 

There  is  need  lor  more  senior  ott ii  er 
involvement  in  the  earlv  stages  ot  re 
quirements  del init ion 

Too  much  supporting  document.! 
tion  is  required  tor  requirements 
doi  umenfs 


AR 


In  February  l°8o.  a  final  joint  work¬ 
ing  group  preceded  a  general  officer 
consensus  meeting  on  March  7,  l°8o, 
wherein  the  revised  Army  Regulation 
71-°  was  approved  with  minor 
changes.  The  regulation  was  forward¬ 
ed  to  key  agencies  March  27.  OSes  tor 
implementation  in  drat t  form  pending 
final  publication.  The  regulation  has 
completed  final  slatting  and  is  in  the 
lina!  edit  process  prior  to  publication. 


Results  of  the  Reform  Process 

Concurrent  with  issuing  the  draft  tor 
implementation.  TRADOC.  Head¬ 
quarters  issueil  a  letter  ot  instruction 
'  l  Ob  Combat  Developments  Tailor¬ 
ing  Process  -  Management  ot  Re¬ 
quirements  Documents  and  shortly 
theicatter  a  standing  operating  pro¬ 
cedure  -SOP'  Management  ot  Re¬ 
quirements  Documents  Within  HQ 

IRAIXX  I  hese  two  documents  are 
inti  lim  guidance  tor  procedures  to  be 
utilized  in  developing  and  slatting  re 
uurements  doc  uments  using  the  new 
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; :  cmergs  doc  uments  the  hnal  draft  s 
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One  Isev  change  in  the  new  AK  71-°  p 
! ' 1  r  :v,  uii/at  ion  ot  the  battlefield  f- 
ve.opmcnt  plan  HP!’  as  primarv  s 
:\e:  ot  the  ettorts  ot  the  technology  v 
se  Previous!  v  considered  a  j 
e.Ai  \\  iloiunren;  the  BPP  non  v 
o>.  ides  the  basi-  tor  prioritization  ot  ( 
tv.',  rescan,  h.  development  and  ae- 
tsit.ov  KPA  programs  in  the 
par: men:  o:  the  Armv  long-range 
\\  plan  1  RKPAP  The  l.RRDAI’  ' 
: : ■  t  Atv.v.  15-vear  prioritized  KPA 
ir  .ino  ;s  t.ne  starting  puce  for 
Id  mg  hue  KPA  portion  ot  the  pro-  y 
am  ob  e.me  memorandum  ■  POM'.  X 


l  he  new  recti:. atam.  leiogm/e-'  ttu 
streamlined  acquisition  initiatives  in 
AK  70-1  and  the  taet  that  tailoring  ot 
the  lite-cycle  system  management 
model  1  (  sMM  is  now  the  norm 
rather  than  the  exception.  A  chart 
showing  the  streamlined  process  com¬ 
pared  to  the  traditional  process  is  at 
Figure  2.  In  order  to  meet  time  lines  ot 
streamlining  initiatives  the  1.01  pro¬ 
vides  a  goal  that  the  required  opera¬ 
tional  capability  be  approved  within  2 
vears  after  the  initiation  ot  the  opera¬ 
tional  and  organizational  plan.  The 
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new  regulation  eliminates  the  letter  re¬ 
quirement  and  letter  ot  agreement 
leaving  the  operational  and  organiza¬ 
tional  plan  and  the  required  opera 
tional  capability  as  the  onlv  re¬ 
quirements  documents  needed  for  a 
normal  system  acquisition.  This 
change  simplifies  the  process 

The  new  operational  and  organiza¬ 
tional  plan  is  the  onlv  requirement  tor 
the  advanced  development  to. 3 
category  ot  funding  tor  all  systems  ex¬ 
cept  DOI3  major  systems  and  HQD A 
designated  acquisition  programs, 
A  IMSNS  is  required  in  addition  to  the 
operational  and  organizational  plan 
tor  program  initiation  ot  Department 
ot  Detense  major  systems.  This  change 
means  that  the  operational  and 
organizational  plan  must  serve  the 
function  previously  played  by  the 
operational  and  organizational  plan 
and  the  letter  ot  agreement.  This  dic¬ 
tates  that  the  document  be  more  detail¬ 
ed.  It  is.  however  limited  by  regula¬ 
tion  to  10  pages.  To  avoid  excessively 
restrictive  detail  operational 
characteristics  are  described  in  broad 
bands  ot  capability:  e.g.  a  capability 
is  required  to  defeat  a  certain  degree 
c't  armor  protection  at  X-Y  meters. 

The  required  operational  capability 
is  the  requirements  document  tor  the 
full-scale  development  portion  ot  the 
life-cycle  system  management  model. 
It  is  required  to  enter  into  the 
demonstration  and  validation  phase  ot 
the  lite-cycle  system  management 
model  for  Department  of  Detense  ma- 
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'or  systems  and  Headquarters  Depart¬ 
ment  ot  ;hc  .-\rniv  designated  acquisi- 
>n  programs  Provisions  are  made 
tor  r!ie  fact  that  information  normally 
will  not  he  available  tor  a  full  required 
operational  eapahilitv  at  that  point  it 
!-■  recognized  that  the  required  opera¬ 
tional  eapahilitv  must  he  updated  anel 
approveei  be  Headquarters  Depart¬ 
ment  ot  the  Armv  bete’re  entry  into 
tull-scalc  development.  The  new 
regulation  permits  tailoring  ot  required 
anahsis  to  meet  new  time  lines  which 
are  goals  not  requirements  establisheei 
in  the  L.Ol  2p  weeks  tor  an  opera¬ 
tional  and  organizational  plan,  and  3o 
weeks  tor  a  required  operational 
capability . 

khe  revised  regulation  establishes  a 
new  document  Operational  Needs 
Statement  iONSe  providing  an  oppor- 
tunitv  tor  commands  and  agencies  out¬ 
side  the  combat  and  materiel  develop¬ 
ment  communities  to  initiate  the  com¬ 
bat  development  process  by  stating  a 
user  -  operational  need  tor  a  materiel 
soitp’on  to  correct  a  deficiency  or  im¬ 
prove  a  capability  which  impacts  upon 
mis-ion  accomplishment.  Although 
not  a  requirements  document,  it  will 
' '(ten  lead  to  one.  " 

Capstone  requirements  documents 
are  ret  og.ni/ed  and  enc  ouraged  by  the 
revised  regulation.  These  operational 
and  organizational  plans  and  required 
operational  capabilities  are  prepared 
tor  families  ot  materiel  to  encourage 
using  the  same  or  similar  maior  com¬ 
ponents  such  as  common  chassis  tor  a 
family  ot  veilic  les 

(  sing  the  new  stalling  letter  ot  in¬ 
struction  procedures  tor  document 
slatting  are  shortened  and  streamlined 
and  are  characterized  bv  tront-end 
•-v > ' :  !■  and  one -time  formal  staffing.  • 
i  'i  'll",  a  men  ‘  is  p  re  pa  red  bv  the  pro- 

. .  -a  :  . u k  ami  | :.; ,1  v  st atted  inter- 

i.a.  rv  "-.it  t  font-end  work  . 

: .  i  for  ,  i enments  and  u >n. 

:  n  -i  ■  v'l  ‘;mi  t.  >rmal  si  ailing 
:  V  11  X  ;  lead, matt.  rs  1  RAI  XX 

■  •'  .  •  :  s  I  KADOC 

•  •  o  .  \'.1(  ;  >.  adquafers  AMC 

•  I  •  '  ■  '  ■’  S..P, 

■  o  :  r  -r;:MU  \1A(  C  >M  A 
•  >  •;  •  •  i  "  i  -  -  • : .  i  • .  • :  I'a  do.  ument 

:  ••.:  ;  ...  ,  ••(-,  :  MAC  (  )\I, 

:  ‘  \  .  . .  .  Mia'.irv  I  tattle 


Alter  tins  one-time  torma!  stalling 
a  ioint  working  group  meets  and  at¬ 
tempts  are  made  to  accommodate 
comments  and  disputes  1  he  doc  ument 
is  revised  based  on  the  joint  working 
group  and  tot  warded  (torn  the  m  hoo! 
commandant  to  AMC  Headquarters 
and  to  FRA  DOC  Headquarters 
through  integrating  centers  tor  ap¬ 
proval.  The  document  is  reviewed  by 
the  FRADOL  RRC  The  RRC.  must  in¬ 
clude  representation  from  the  propo¬ 
nent  school  AMC  Headquarters  and 
the  AMC  proponent  maior  subor¬ 
dinate  command.  Alter  the  RRC  com¬ 
ments  are  incorporated  by  the 
TRADOC  DCSCD  action  officer,  it 
minor:  or  by  the  proponent  school  it 
maior.  The  document  is  forwarded  to 
the  proper  level  tor  approval. 

The  change  in  approval  authority  is 
one  major  improvement  ol  the  new  re¬ 
quirements  process.  Whereas  I.OAs 
formerly  had  to  be  approved  by  Head¬ 
quarters,  Department  ot  the  Army,  lor 
all  programs  with  systems  advanced 
development  costs  exceeding  S25 
million,  now  all  operational  and 
organizational  plans,  which  replace  the 
letter  ot  agreement  tor  most  systems, 
can  be  approved  by  the  TRADOC 
Commander.  Department  ot  Detense 
major  systems  require  a  justification 
for  major  system  new  start  to  be  ap¬ 
proved  bv  the  Secretary  ot  Detense. 
The  ROC’s  tor  programs  costing  less 
than  S 1 00  million  in  RDTI:  and  or 
S500  million  in  procurement  may  be 
approved  by  the  joint  signature  ot  the 
FRADOC  Commander,  and  the  AMC 
Commander.  This  approval  level  in¬ 
cludes  most  systems  and  will  shorten 
the  approval  process  The  ROCs  more 
than  the  SlOO  million  and  S500  million 
limits  must  be  approved  by  Head¬ 
quarters.  Department  ol  the  Army 

Another  maior  improvement  in  the 
materiel  requirements  dot  ument  pro 
less  is  establishment  ot  the  materiel 
development  program  review 
established  bv  the  April  10  l°8o  SOI' 
and  promulgated  to  T  RAIXX  si  hools 
by  message  It  requires  that  each  pro 
ponent  school  briel  the  IX  St.  D 
I  RAI  K  X  on  a  quarlcrlv  Ki-n  about 
the  status  ot  developmental  programs 
during  the  review,  programs  are  given 
a  status  rating  ot  green  amber  or  ted 
bv  managerial  decision.  It  a  program 
talk  behind  its  protected  tins  line 
be  ailse  ot  Ol  "  ‘i'.i! ;.«:  ••  .e.  ! •» 

i  led  amber  ot  red  1  he  I  RAI  H  X 
Headquarters  rr<  ords  t hese  p: o-.  ,  : 
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materiel  requirements  document  pro- 
c  ess .  It  they  are  effective  major  pro¬ 
cess  time  sav  ings  should  be  realized 
and  I  RADOC  should  be  able  to  divert 
many  man-hours  to  other  activities. 

Effectiveness  of  System  Reforms 

Because  reforms  are  in  a  draft  stage 
and  implementation  began  in  late 
March  l°8o,  findings  oi  this  assess¬ 
ment  ot  system  reforms  arc- 
preliminary.  In  order  to  accomplish 
the  assessment.  1  interviewed  person¬ 
nel  at  Headquarters,  Department  ot 
the  Armv:  Headquarters,  Training  and 
Doctrine  Command:  Headquarters 
Army  Materiel  Command:  L'.S.  Army 
Combined  Arms  Center:  L'.S.  Army 
Infantry  School:  L'.S.  Army  Engineer 
School:  and  U.S.  Army  Air  Defense 
Artillery  School.  Initially,  an  attempt 
was  made  to  get  hard  data  on  whether 
projects  were  meeting  the  newly 
established  time  goals.  Because  there 
were  so  many  exceptional  cases  and  so 
few  projects  actually  started  since  in- 
troduc.ion  ot  the  new  system  became 
effective,  this  course  of  action  was 
abandoned  as  impractical.  Findings, 
therefore,  are  not  only  preliminary  but 
subjective.  Note  that  schools  and  in¬ 
tegrating  centers  other  than  those  in¬ 
terviewed  may  have  had  different  ex¬ 
periences.  In  spite  ot  these  shortcom¬ 
ings  many  items  ot  interest  arose 
which  should  be  useful  tor  making  the 
new  system  work 

On  the  positive  side: 

The  increased  approval  authority  at  ! 
lev  els  below  I IQDA  is  seen  as  a  maior  i 
gain.  It  meets  the  users  need  by  get-  | 
ting  doc u merits  approved  more  quic k -  j 
Iv  and  ettic  ieritlv 

It  serves  to  Hear  the  logtam  ot  J 
Joiunients  ,it  the  overworked  Armv 
Vat!  Beta. me  ol  -hi  om  deep  stalling 
on  the  Armv  stall  the  probli  ot 
moving  mam  .ioi  i.mi  nN  lor  qua  !>  ap- 
prov.il  was  unsolv  able  Now  the  stall 
i  an  concentrate  or>  tewe:  and  more  ;m  ^ 
pi  'ft  ant  sti-”w 
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i.i : 1  man  Pule:!!?  t  s  Arnn 
. "  ::-r  !  Yput\  x  nut  ot  Vat  !  r . 1  r 
:  -  >■>  and  !  ’Ians  Interx  sew  A  I 
t :  :  °8r 

Atlanta  “seminar  i  ih  U'C'  » >n 
qui'ition  lnitu»:\ es  Mrmi. 
P.  :  r.'.'rn:,  ';'  ,r:.i  A, 

Mav-lune  1J82  p.  5. 

inter \  uas  with  IK.  Robert  I 
I X  X .  tor  R  1  Xv  A  A >  **;i. 
s  •:  IV'Vi.'," mv*;.*  i in. I  Aiiiui-: 
A !u<u; : >.v  Ian ua r v  Tebr uar\ 

p  20 

It-n  William  R  Rk  hardson 
ssions  with  Mr  Ambrose  on  the 
re  merit'  Protest  Memoran- 
l  XAlRAPtX  Ft.  Monroe 
.  CX  tobor  1°84 

i-M  isMvi  Interview  (den. 
A  R  Thurman  Arm\  Via 
o:  Start  A >  »t v  K, •-,•>>> ,  h 

a*;,/  Aitiui'itu'i:  Maya- 
v  'UTnh  : -I  X’t  ember  084  p  2 

hardson  Memorandum 

vli'i  !i;u  notes  on  (.  olonel-leve! 
1  .t"'.;p  on  Materiel  Acquisition 
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I  he  Arnn  s  i  n  30  and  o0k\\ 
.aeneiati'i's  have  been  refitted  with  a 
nen  an,l  miprovei!  yovernoi  adapted 
toi  militai A  use  I”,  the  1  loop  sappor  ! 
e  ommanii  belvoir  Rl  Tk  1  e  entei  It  b 
more  reliable  easier  to  maintain  less 
expeinne  and  lighter  in  weight  than 
the  elei  tro-I'iVilrauhi  s\stem  previous- 
!v  used 

I  ests  showed  th.lt  the  new  poveinoi 
will  run  an  average  ol  8  5° 2  hours  be¬ 
tween  t.i'lures  lompauii  to  '  88“ 
hours  tor  the  elei  tro-livdiaulu  unit 
On  the  average  the  new  qovernor 
weighs  88  pereent  less  Lnuineeis 
estimate  the  new  governor  will  save 
the  Arm v  nearlv  Si  million  dui  in. a  t he 
next  3  tears  Ne.nK  2  400  generators 
already  in  the  Aram  mventorv  were 
retrofitted  new  >:i  iterators  will  be 
equipped  with  the  governor  when 
built  ■ 


1  I  he  l  ^  Arm',  -nved  more  than 
I  S870  000  as  a  result  ot  a  value 


riu 

lort  Rii 

i  hie 

Md 

rPip!i'\  n 

I  he  At 

PH 

cx 

lames  p 

Mi  <  hi :  re  < 

I'mmunu  .»*  lop-  ■ 

*■*  Mr 

i  yv 

spei  hills! 

Ot  t  K  t  i 

>r  ! he  1 X  put v  X  h u  t 

si;r  vr  \  ;:ii: 

ni 

i  ot  S(,it|  to 

r  Plans 

<irul 

X  ’•per  a!  ions  to: 

i’h:'’'  "• 

Art 

1.  1 

tire  ,  th  Nunn!  (  e 

md  heailiiuat 

.  it  !  ‘  A  k'. 

1  % 

I  :r-.:!  V.  rsion  ot  AR  2!  J 
■  V.*  morututum  Memo 
MOl'A  PAMxMPK 


!  tereii  at  lor!  Kite  hie  teiommended 
that  as  part  ot  the  i  on!  met'.ta!  1  o. 
'st.m  ,  telephone  modernization  pro 
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in  re  is  a  national  c  iMM-nvui  that 
produt  ! ! vit\  improvement  is  .in 
important  issue  tor  the  coming 
decade  l  niil'tl  States  !iuli:slr% 
mast  improve  output  and  quail 
t\  to  continue  a  leading  torn'  in  world 
u'liinuTu'  l  or  people  in  hoy eminent 
t he  requirement  to  improve  perform 
arm  is  urgent  to  minimize  the  tax 
burden  and  to  provide  produe  i i yi! \ 
leadership 

Productivity  Measurement 

One  i  haliengmg  requirement  ot  pro 
liuitivitv  improvement  programs  is  to 
design  methods  lor  measurement  ot 
produe  tiYitY  c  fiances.  I  he  c  hallenge  is 
parf.v  u'.arU  dittuuit  when  the  activi- 
whose  producin', t\  is  being  meas¬ 
ured  prodia.es  intangible  support 
rathe:  than  a  tangible  produet  or  a 
measurable  serene  Sta  h  ai  tivities 
hereirtaiter  willed  support  activities  m- 
elude  m.-st  administi  ative  and 
ertitmeer urn;  organizations  1  bough 
s..ppo::  ,k  :  Wittes  produce  tangible  en- 
■  a  a  s  ,m  ret'orts  drawings  and 

mi  in  ii. i  :t  often  >  dittuuit  to 

i  ham  >  to  p:  odm  :  i\  it \ 
ut  ■  a..-!  a  t  ta  •  i.a1  ton  m  : \  pe  and 
■  u  e  u  . :  \  ot  s;a  :  or  ium  i.’a!  tops 

a:  at  tt  mpr  to  ir'e.Kin  support  at 
'.■.'as  pr  ■  ui.a  ‘ :  v  :t  measu  rement  s 

ir*  •> <  ’v.iclt  ■'?  pin  it'  :rui: 

a'  -•  -  t  t'er  ti  :  rriarit  e  nit  lading  tor 
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that  while  the  system  proposed  herein 
is  designed  to  measure  t  ustomer  satis- 
itKtion  it  is  in  reality  measuring  the 
activity  s  performance  against  separate 
tasks  that  the  activity  must  accom¬ 
plish.  A  student  s  report  card  is  a 
simple  analogs  It  measures  student 
performance  not  tear  her  satisfaction 
C  .. stomi't s  ■  and  le.u  her  ~  at*  h:o..gh: 
into  the  evaluation  process  because 
thee  are  at  least  tor  some  tasks  KM 
ci ci.il it iccl  to  evaluate  perti'rmance  d  o 
be  sure  there  will  be  tasks  where  per¬ 
formance  is  a  matter  ot  record  and 
c  ustomer  ec  alu.it  ion  is  r.<  >1 
needed  lor 
the  attendance 
rei  ord  i 


this  categorv  hor  commercial  activi¬ 
ties  profit  loss  is  such  a  parameter:  tor 
public  sector  activities  adherence  to 
budget  and  or  manpower  limits  are 
tacts  ot  record  needing  no  customer 
evaluation  Indeed,  by  structuring  a 
number  ot  performance  parameters  in 
wavs  that  make  them  subject  to  infor¬ 
mal, on  ot  iciowl  c.g.  ratio  ot  pro¬ 
grams  under  cost  to  programs  over 
cost  i  customer  satisfaction  tan  with¬ 
out  undue  complication  ot  the  process 
be  one  ot  a  mix  ot 
productivit>  per- 
torma  nee 
indicators. 


The  Ui-torntr  satisfaction  prod... 
:ivitv  :m*jMirfnu’nt  system  proposed 
'’i  re  begins  with  the  generally 
nremise  that  each  oryani/at  mal  unit 
exists  to  '•up port  or  M'rvi' c  ustomer  s 
A  second  premise  ;s  that  per!  or  mam  e 
is  the  same  tor  our  purpose  as  pro 
.hie ! :  vtt  v  a  third  premise  pt-rf i >r 
!!ianu  improvement  has  economic 
v  alue  and  van  he  ejuantiticd  monctari  - 
K  a  fourth  premise  c  ustomer  satistai 
tion  v  an  he  measured  w  ith,  relative  ob 
ev  1 1 v ! ! \  a  titth  premise  a  measure 
merit  ot  customer  satisfaction  is  the  in¬ 
tegral  ot  all  other  performance 
measLirements  Buildmy  on  these  1 
propose  a  measurement  s%  stem,  tor 
i.  onsideration 

I  he  performing  organizational  unit 
enters  into  an  informal  agreement  with 
•  u stonier1 '  about  the  service  it  pro 
v  vies  Where  possible  the  perfor¬ 
mance  contract  should  contain  pert  or 
mariie  standards  preferable  but  not 
nec c-ssari!v  ciuantit.it ive  attains!  which 
performance  can  subs.-ciiientlv  he 
i  v  a  mated 

based  o:i  the  performance  c  ontr.u  i 
a  i  U'tomer  report  card  a  sponsor 
satistai  turn  inde'  is  nettotiated  be 
tween  pertortmne  orttani/ation.il  unit 
and  customer  s  The  report  card 

u-.aht  include  items  like  timeliness 

u  ,,il;'v  and  adherent .  to  budget 
\ |o:  e  'i; h;cc  t .  ve  tac  tors  ;u  c  oopei  a 
t:v«-ness  and  responsive  r*ess  vou!d  be 
m.  ludeci 


measured 
- base 


®  100 


Anpuaih.  o:  more  rrecjucntiy  the 
s  v\  i ; '  :  u-  prr  P  >MV.ir.i  r  or 
p<  rt or  >;n:t 

.ii rmi  -spopsto  s.iMstcii  •  ;•  ■ : •  .ruit  v  to: 

::m‘  \  rtA;..;mr  rrpi *rt ::v.  c;...,i!!tr.\  .it 
;o!'s‘, i;w  'vot:\.i!  or*  .»p0.  rvfi-r 
r »  , : ’  * v  orrro!  cirPMP.i 


Let  us  assume  that  performance  Is  100  percent  in  the  base  year  At  year  1 
measured  performance  rises  2  percent  to  102.  so  the  base  rises  by  t  percent 
to  101  At  y8ar  2,  measured  performance  rises  1  percent  to  103  (which  is  2 
percent  above  the  year  1  base)  so  the  base  rises  by  1  percent  to  102  At  year  3. 
measured  performance  drops  4  percent  to  99  which  is  3  percent  less  than  the 
year  2  base:  the  base  drops  by  1  1/2  percent  to  100  5 
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nhanced  timeliness  and  data 
quality  .  znefit  program 
managers  when  contract  per- 
tormance  reports  are  submit¬ 
ted  via  electronic  transmis¬ 
sion.  A  system  tor  electronic  reporting, 
Associate  Contractor  Report  Submis¬ 
sion  Interface  Subsystem  (ARIS),  has 
been  implemented  at  the  Ballistic 
Missile  Office  (BMO),  Norton  Air 
Force  Base,  Calif _  Using  this  system, 
contractors  can  transfer  contract  per¬ 
formance  data  directly  to  computer 
data  storage  at  the  BMO,  where  it  is 
processed  to  provide  briefing  docu¬ 
ments  tor  contract  analysts.  The 
system  uses  communication  protocols 
which  detect  and  recover  from  noise 
interference  during  transmission:  thus, 
standard  voice-grade  telephone  lines 
i  an  be  used.  Performance  evaluation 
tor  contract  reports  electronically  sub¬ 
mitted  can  be  completed  about  3  weeks 
sooner  than  for  those  submitting  hard- 
i  opv  reports  by  mail 

I  he  fictional  piogiam  control 
analcst  reviews  c  ale  illations  of  recent 
cost  performance  report  data  paying 
attention  to  the  contractor  s  estimate- 
at  completion  values  Realizing  the 
general  s  briefing  is  within  the  hour  on 
tht  internal  program  office  estimate  to 
.  "mp'ete  the  proiei  t  and  thee  ontrac  - 
tot  s  ,  ^responding  value  the  analyst 
ponders  mane  questions  Why  are 
•  t  ere  different  es  in  the  numbers’  How 
.!>  •  I  rei  orn  he  values? 

1  f'e  report  shows  data  to  be  30  days 
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recognized  the  problem  of  timely  data 
early  in  the  development  of  the  Peace¬ 
keeper  program.  Contractors  require 
X  amount  of  days  after  the  closeout 
ot  the  accounting  month  to  collect, 
analyze  and  report  cost  performance 
values.  Mailing  several  copies  ot  an 
often  300-page  report  consumes  time 
and  creates  enemies  in  the  corporate 
mail  room.  Automated  analysis  ot  the 
report  requires  initial  loading  ot  the 
data  and  an  error  check  routine  before 
substantive  data  analysis. 

In  1°83  the  BMO  Systems  Informa¬ 
tion  Directorate  set  out  to  develop  a 
means  to  electronically  transmit 
cost-performance  report  data  via  or 
chn.irv  telephone  lines  from  the  con¬ 
tractors  tai  ility  to  the  BMO  cost 
analysis  computer  Obviously  the  at¬ 
tempt  w  as  to  i  ut  out  the  middleman 
to  s.ivi  time  and  improve  aciur.iiv 
Figure  i  shows  data  flow  from  i  ontrac 
tor  to  the  BMC)  via  conventional  anil 
automated  paths  Development  ot 
HMl  '  sof 1  w  a  re  and  prot  cd  rites  initial 
w  a s  ai  i  oniplehed  tin  Si  iern  e 
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transmission  system.  Today.  BMO 
receives  the  majority  of  its  contractor 
cost-performance  report  data  via  this 
computer-to-computer  interlace  sys¬ 
tem.  As  a  result,  the  program  control 
team  can  he  ready  to  brief  senior-level 
management  on  cost-performance  data 
within  2to  days  after  the  closeout  of  the 
accounting  period.  This  is  a  tar  cry 
from  the  aforementioned  example. 

Requirements  for 
Automated 
Reporting  System 

Initiating  an  automated  reporting 
system  should  impose  minimal  burden 
on  contractors  using  it.  Since  cost- 
performance  reporting  normally  will 
he  an  established  activity  ot  the  con¬ 
tractor  using  the  system  should  not  re¬ 
quire  extensive  modifications  of  ex¬ 
isting  accounting  and  management 
procedures  or  require  expensive  sett 
ware  or  hardware  Transmissions 
should  be  possible  from  personal-! vpe 
computers  equipped  with  modems  us¬ 
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accumulate  on  the  line  between  trans¬ 
mission  ot  records  are  discarded  until 
the  start-ot-data  sequence  is  detected 

Finally,  standard  redundancy  tech¬ 
niques  are  used  to  detect  errors  caused 
by  m  ise  during  reception  ot  the 
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interterence  ettects  (  noise  )  which  can 
corrupt  data  transmitted.  The  system, 
thc-rdore.  will  require  a  transmission 
protocol  detecting  noise  and  allowing 
tor  recovery  from  its  effects.  If  the  line 
connection  is  broken  before  a  trans¬ 
mission  is  complete,  recovery  pro¬ 
cedures  should  allow  resumption  of 
transmission  without  requiring  retrans¬ 
mission  ot  data  already  received. 

It  is  important  that  system  users 
have  convenient  access  to  qualified 
support  personnel  who,  in  turn,  have 
status-monitoring  capabilities  allowing 
them  to  respond  to  queries  and  suggest 
corrective  actions.  It  is  particularly  im¬ 
portant  that  these  support  functions  in¬ 
clude  test  facilities  so  that  new  users  of 
the  system  'or  current  users  wishing  to 
test  software  or  hardware  modifica¬ 
tions'  can  make  test  transmissions 
without  interfering  with  day-to-day 

t  ransmissions 

See  unity  of  data  is  a  ma|or  concern 
for  the  contractor  submitting  reports 
and  the  agency  receiving  them  The 
idea  ot  a  telephone  link  to  a  computer 
system  supporting  sensitive  data  bases 
raises  important  questions  Could  a 
competitor  gam  access  to  cost  and 
funding  data?  L  ould  data  tor  a  com¬ 
plete  program  be  obtained  bv  a  hostile 
part'T  [he  system  should  allow  only 
one  wav  program-fo  program  coni 
m u n n  at 
prec  lude 
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The  ARIS  System 

The  electronic  transmission  system 
satisfying  these  requirements  was 
developed  for  report  management  for 
the  Peacekeeper  ICBM  development 
program.  It  now  is  being  applied  for 
Peacekeeper  production.  Small  ICBM 
development,  and  Advanced  Strategic 
Missile  System  development  manage¬ 
ment.  This  system  is  called  ARIS 
(Associate  Contractor  Report  Submis¬ 
sion  Interface  Subsystem)  and  provides 
electronic  submission  capability  for 
cost  performance  reports  (CPR)  and 
contract  fund  status  reports  (CFSR). 
The  ARIS  is  resident  on  a  Prime  750 
computer  system  which  supports  the 
cost  performance  analysis  and  data 
subsystem  (CPADS). 

The  transmission  protocol  used  by 
ARIS  is  relatively  simple  but  effective 
tor  erri  r  detection  and  recovery  at  low- 
transmission  rates.  Three  techniques 
are  used. 

First  a  report  is  sent  as  a  series  ot 
short  records  ( less  than  250  characters 
each).  It  an  error  is  detected  during  the 
reception  ot  a  record  the  reception 
program  instructs  the  transmission 
program  to  resend  that  record  Thus, 
it  is  not  necessarv  to  have  noise-free 
line  conditions  during  the  time  it 
would  take  to  transmit  the  complete 
report  as  one  record 

'sn  ond  i  at  h  retord  is  cm.ip 
suiated  :  e  a  start  sequence  prei  edes 
the  at  tual  data  and  an  end  sequent  e 

follows  it  noise  characters  that  may 


data  flow  are  indicated  in  Figure  2.  The 
system  s  goal  is  data  transfer  for  a  con¬ 
tract  to  the  CPADS  data  base  tor  use 
by  other  systems  preparing  analysis 
and  briefing  documents.  Auditing  and 
archiving  processes  accompany  the 
passage  of  a  report  through  the  system 

The  Receiver 

The  reception  program  operates 
continuously  as  a  background  process 
on  the  host  computer.  It  periodically 
checks  the  communication  line  and 
when  a  carrier  signal  is  detected  the 
program  initiates  data  reception.  The 
receiver  communicates  with  the  trans¬ 
mission  program  running  on  the  con¬ 
tractors  computer  by  sending  one  ot 
three  2-character  control  sequences. 
These  respectively  instruct  the  trans¬ 
mission  program  to  send  the  next 
record,  to  resend  the  previous  (i.e.  an 
error  was  detected),  or  that  the  receiver 
is  terminating  the  transmission  (i.e..  a 
non-recoverable  condition  exists,  see 
Table  II.  A  log  is  maintained  ot  all 
control  sequences  sent  and  error  con¬ 
ditions  detected;  the  data  received  are 
stored  in  a  temporary  file.  Typical  net 
reception  rates  are  5.000  to  5.500  char¬ 
acters  per  minute.  This  is  about  ten 
times  the  rate  at  which  an  expert  word 
processor  can  input  data. 

When  a  transmission  is  terminated, 
the  program  verifies  the  contractor 
password,  contract  number  and  report 
type  so  that  the  data  can  be  transfer¬ 
red  to  the  proper  permanent  tile  tor 
subsequent  processing  When  this  is 
done,  the  receiver  spawns  several 
parallel  processes  and  returns  to  its 
line-monitoring  mode  These'  processes 
handle  the  archiving  ot  the  data  as 
received  produce  a  hard  copy  (nr  use 
by  data  entry  personnel  and  initiate 
the  data  verification  process 

■  Lieutenant  Colonel  Abate  is  the 
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•BACKGROUND  PROCESSES 
••BATCH  PROCESSES  {Via  Operator  I 


The  Verifier 

The  successful  termination  of  a  data 
transmission  means  that  the  report  has 
been  received  error-tree  in  the  sense 
that  no  loss  or  spurious  introduction 
ot  data  occurred.  The  verification  pro¬ 
gram  s  task  is  to  analyze  the  received 
data  tor  content,  and  to  produce  a 
report  showing  inconsistencies  in  data 
that  must  be  corrected  before  being 
committed  to  the  data  base. 


record  type:  the  program  checks  if  they 
are  present.  Cost  performance  data  are 
typically  related  by  line-item  name  to 
a  master  work-breakdown  structure 
( WBS);  line-item  names  are  checked  to 
make  sure  they  agree  with  the  current 
WBS  for  the  contract.  Finally,  checks 
are  made  for  numerical  consistency. 
The  output  from  this  module  is  a  hard¬ 
copy  exception  report. 


Records  in  the  report  contain  a 
single  letter  code  identifying  them  by 
type.  The  first  verification  checks  each 
ecord  to  see  it  it  contains  the  correct 
lumber  ot  data  fields  for  that  type  of 
record,  and  that  these  are  ot  the  cor¬ 
ed  data  type  -  text  or  numeric  .  Cer¬ 
am  line  items  are  required  tor  each 


Ideally,  this  report  will  show  no  ex¬ 
ceptions.  Should  problems  exist,  it  will 
be  necessary  to  modify  the  data.  The 
course  ot  action  followed  will  depend 
on  the  severity  of  the  problem.  Data 
entry  personnel  are  authorized  to  cor¬ 
rect  obvious  errors  in  spelling  or  punc¬ 
tuation,  but  substantive  modifications 


Invalid  Character 
Parity  Error 
Checksum  Error 


-  Line  Disconnected 

—  Record  Loss 


--  Time  Out 


—  Ten  Consecutive  Requests 
for  Retransmission 


Invalid  character  One  not  in  the  allowed  AHIS  set 
Time  out  3  seconds  of  inactivity  on  the  line 

Record  loss  Successively  received  records  don't  have  successive  numbers 
Parity  and  Checksum  errors  indicate  line-noise  interference 
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1  Program-to-Program  Communi¬ 
cation 

Data  are  sent  by  a  program  running 
on  the  contractor's  computer  to  a  pro¬ 
gram  running  on  the  BMO  computer. 
This  means  the  contractor  does  not 
log  on  to  the  BMO  computer,  nor 
does  the  BMO  Tog  on”  to  the  contrac¬ 
tor's  computer.  Neither  computer  can 
issue  commands  to  the  other's  oper¬ 
ating  system. 

The  two  programs  establish  a  com¬ 
munications  link  solely  for  data  trans¬ 
mission  purposes  and  the  receiving 
program  accepts  data  only.  With  this 
scheme,  data  bases  and  operating  sys¬ 
tems  on  either  end  are  insulated. 

2.  One-way  Data  Transfer 

Data  received  by  the  AR1S  system 
are  not  echoed”  back  to  the  sender,  i 
This  means  data  flow  only  one  way: 
from  the  contractor  to  the  BMO.  The 
receiving  program  sends  only  instruc¬ 
tions  to  the  transmitting  program. 

- 

can  be  made  only  with  the  approval  of 
the  program  control  analyst.  If  prob¬ 
lems  are  severe,  the  analyst  will  request 
the  contractor  to  resubmit  the  report.  [ 

Alter  modification,  the  report  is  sub-  | 
mitted  again  to  the  verification  proc¬ 
ess.  The  report  remains  in  this 
modify  verify  cycle  until  all  exceptions 
are  resolved,  and  it  is  ready  to  be  ( 
transferred  to  the  data  base. 

The  verification  program  requires  ■ 
considerable  computer  resources, 
especially  processing  time.  For  this 
reason,  it  runs  as  a  background  proc¬ 
ess  so  that  operators  may  proceed  to 
other  tasks  while  the  verification 
proceeds. 

Data  Base  Updater 

The  ARIS  transmissions  are  received 
in  rather  tree  tormat  in  o  rdiT  to  spa  re 
the  sender  the  overhead  ot  fixed  record 
and  field  lengths  Data  bases  how 
ever  impose  strict  formatting  re 
quircments  on  data  which  is  trans¬ 
ferred  to  them  I  he  update  program  s 
tirst  task  is  to  reformat  the  data  to 
C  PADS  spec  it  ic  at  ions  I  he  data  is 
then  committed  *o  the  data  !\i--c  and 
is  available  tor  analvsw  ]•<  •.•■a,  ‘.w 
is  [i  •  til<'  a  c  opv  ot  the  ;’li st it  :i\i  trails 
mission  in  the  arc  hives 


MONTH  1 


MONTH  2 


MONTH  3 


ELECTRONIC 

TRANSMISSION 


TIME 


MANUAL 
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Close  of 

Accounting  Period 


I. 


BRIEFING 


RD  B 
V 


4 - 22  Days - ♦ 


ft  ft  tt 


BRIEFING 


R  RE  E  V  D  B 


End  of  Report 
Preparation  Period 


B  =  Briefing  Document 
D  =  To  Data  Base 
R  =  Received 


RE  =  Received  Data  Entry 
E  =  Data  Entered 
V  =  Verified 


Utilities 

Certain  program  modules  ac¬ 
complish  tasks  that  are  not  apparent 
to  the  contractor  submitting  the  report. 
These  utility  programs  carry  out 
auditing  and  archiving  functions  and 
enable  the  data  entry  and  support 
statts  to  perform  their  functions 
efficiently. 

[Tata  modification  is  done  using  a 
special-purpose  editor  specifically 
designed  to  handle  data  in  the  ARIS 
format.  This  has  proved  more  efficient 
than  general-purpose  text  editors. 

Maintaining  archives  is  an  impor¬ 
tant  function  of  the  system.  Immedi¬ 
ately  following  reception,  a  copy  of  the 
as  received  report  is  transferred  to  a 
file  in  an  archive  area  of  disk-storage. 
Reports  tor  a  t>-month  period  are  re¬ 
tained  in  this  archive.  Reports  for 
earlier  periods  are  stored  on  magnetic 
tape  The  final  version  of  the  report 
following  modification  it  any  is 
similar! v  archived  Should  questions  ot 
data  integritv  arise  these  two  versions 
are  available  for  comparison 


At  each  stage  of  a  report's  progress 
through  the  system,  a  time  stamp  is 
entered  in  a  status  file.  This  status  tile 
can  be  displayed  at  a  terminal  or 
printed.  It  provides  a  snapshot  of  the 
current  month's  activity. 


How  Well  Does  System  Work? 

From  the  i  ontractor  s  point  ot  view 
the  system  has  been  easy-  to  use 
Typical  start  up  times  tor  new  users 
arc  W  to  pO  clays  1  sing  the  ARIS 


system  imposes  no  restraints  on  the 
contractor's  hardware  software  con¬ 
figuration:  however,  the  present  trend 
is  toward  the  use  of  personal-type 
computers  to  transmit  data  files  which 
have  been  down-loaded  from  the  main 
accounting  system  computers. 

The  Ballistic  Missile  Office  believes 
the  improvement  in  timeliness  has  been 
dramatic.  For  example,  time-lines  tor 
two  contracts,  one  using  electronic 
transmission,  and  the  other  manual 
mail  submission  (Figure  31  show  early 
availability  of  useful  management  data 
afforded  by  using  ARIS.  An  additional 
benefit  arises  from  elimination  ot 
manual  data  entry  Decreased  person¬ 
nel  requirements  have  resulted  in  an¬ 
nual  savings  o!  more  than  $200,000. 

In  summary,  ARIS  has  proved  to  be 
a  simplc-lo-use.  secure  electronic 
transmission  system  These  features 
together  wi  its  verification  pro- 
n-dires  have  established  it  .is  an  mi 
portanl  conduit  tor  channeling  man 
.igement  data  to  program  management 
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his  paper  describes  the  Air 
Force  Systems  Command 
AFSO  formal  long-range  plan¬ 
ning  process— Vanguard.  The 
AFSC  is  responsible  tor  all 
research  development,  and  acquisition 
of  all  new  I'.S.  Air  Force  weapons 
systems  The  AFSC  considers  long- 
range  planning  of  critical 
importance  — more  than  an  academic 
exercise.  The  L'SAF  ability  to  continue 
to  confront  successfully  potential 
adversaries  possessing  numerically 
superior  forces  demands  state-of-the- 
art  quality  weapons  systems.  Because 
the  cost  of  research,  development,  and 
acquisition  of  new  systems  is  con¬ 
tinually  rising  while  the  Air  Force 
budget  is  shrinking,  decision-makers 
are  forced  to  be  more  selective  about 
approving  requests  for  funds.  The 
AFSC  realized  that  a  new  approach 
was  needed  to  gain  support  for 
research  and  development  efforts.  A 
formal  system  was  adopted  that  pro¬ 
vides  an  explicit  plan  — rooted  in 
historical  fact  and  concisely  presented. 

Vanguard  was  initiated  in  1978  to 
provide  a  process  for  documenting  the 
using  commands'  needs  and  for  focus¬ 
ing  research  and  acquisition  activities 
in  AFSC  to  satisfy  those  needs.  It  has 
evolved  into  AFSC  s  formal  long-range 
planning  process  and  a  principal  in¬ 
strument  tor  dialogue  with  Air  Force 
warfighting  organizations. 

Vanguard  is  a  highly  interactive 
process.  Development  planners  at 
AFSC  product  divisions  — located 
around  the  United  States— work  with 
Air  Force  major  commands  to  measure 
programmed  capability  against  the 
threat,  and  to  identify  future  capabil- 
ity  shortfalls  in  the  Air  Force's  mission 
areas' Figure  D.  Product  division  plan¬ 
ners  identify  developing  technologies 
and  future  weapon  systems  concepts  to 
meet  those  needs.  The  F1Q  AFSC 
development  planners  integrate  prod¬ 
uct  division  inputs  into  an  overall  plan 
and  in  turn,  review  that  master  plan 
with  all  participants. 

Vanguard  is  a  threat-based  user- 
oriented  'requirements  pull1  planning 
svstem  Fiscal  constraints  based  on 
historical  experience  and  the  postu- 
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Long-range  planning  does  not  deal  with 
future  decisions ,  but  with  the  futurity  of 
present  decisions. 


lated  future  environment  are  imposed 
to  develop  estimates  of  future  funding 
levels.  Vanguard  helps  guide  technol¬ 
ogy  efforts  in  Air  Force  and  industry 
labortories  and  identifies  alternatives 
for  future  weapon  systems  that  will 
meet  the  future  threat. 

Vanguard  is  one  part  of  the  complex 
effort  to  identify  future  Air  Force  needs 
and  thereby  provide  focus  tor  research 
and  development  efforts.  It  is  not  an 
attempt  by  AFSC  to  dictate  to  the  Air 
Force  which  systems  to  buy.  Neither 
is  it  the  final  word  tec  industry  on  ex- 
actlv  which  systems  the  Air  Force  will 
buy  and  when.  However,  it  is  the  most 
intensive  and  comprehensive  look  at 
the  future  of  which  we  in  AFSC  are 


Sub-Mission  Plans 

Mission  area  plans  corresponding 
to  the  ten  mission  areas  in  which  the 
Air  Force  formulates  its  budget  con¬ 
stitute  the  core  of  the  AFSC  planning 
structure  fie.,  strategic  ottense 
strategic  defense  tactical l.  The  foun¬ 
dations  tor  the  plans  are  a  series  ot  sub¬ 
mission  plans  which  provide  a  greater 
degree  ot  detail  Development  planners 
at  AFSC  product  divisions  are  respon 
siblc  tor  the  preparation  and 
maintenance  ot  the-  specific  sub 


—  Peter  Drucker 

mission  plans  in  which  the  divisions 
have  particular  expertise.  For  plans  in¬ 
volving  activities  closely  associated 
with  more  than  one  product  division, 
a  lead  product  division  is  assigned. 

The  sub-mission  plans  are  based  on 
requirements  defined  by  the  com¬ 
manders  and  planners  ot  Air  Force 
operating  commands  — in  conjunction 
with  guidance  from  the  Secretary  ot 
Defense  and  Headquarters  I’.S.  Air 
Force  (Figure  2>. 

Since  the  weapon  systems  we 
develop  and  acquire  end  up  in  the 
hands  ot  operational  commands  we 
want  to  Mart  with  a  firm  appreciation 
ot  what  users  needs  are.  We  do  that 
primarily  by  meeting  frequently  with 
the  users  One  ot  the  most  publicized 
failures  in  business  has  been  Ford  s 
F.dsel  automobile  produced  in  1J57. 
General  consensus  attributes  the  failure 
t ci  incorrectly  forecasting  the  potential 
market  A  case  study  ot  the  lorecasting 
error  revealed  that  the  cur  industry  s 
market  forecasts  did  not  consider  what 
the1  customer  wanted  rather  oniy 
what  the  manufacturers  in  Detroit  had 
concluded  the  customer  wanted 
Vanguard's  goal  w  to  understand  cvh.il 


.  i. 


AIM 


Users 

doc  umen!  those  users  nets 
discuss  then'  with  the  use! 


w  ant 
and 


o 


A.:  at'u :t \  to  liiir!  nation.]  1 

deter:  se  needs  nov\  ami  :n  the  tutuic 
bv  -.wing  -t.mdard  ana!\  tic  a!  computet 
iv.1. It  !',  recognized  and  .u  Li'i’lia! 
:  h : eughou:  the  Air  l  orn'  I  in-  sub- 
plans  spotlight  differences  between  our 
protected  capabilitie-  and  tasks  that 
must  be  accomplished 

Product  die  isum  planners  then  pro 
posi  a  s<.u  v. ■>;  alternative  weapon 
system  solution'  to  lorriv!  each  deti- 
ui't’.iy  Iht'  alternatives  are  based  on 
maturing  technologies  from  both 
I'SAh  iaborati'ries  and  industry 
research.  Throughout  this  phase  the 
plans  are  fiscally  unconstrained. 

because  no  one  pre'cluet  division 
analv.'es  a  total  mission  area  plan, 
eact'.  product  division  s  alternative 
solutions  are  forwarded  to  HQ  AFSC 
tor  completion  ol  the  Vanguard  mis¬ 
sion  area  plans. 


Master  Plans 

The  sub-mission  plans  are  con¬ 
solidated  by  HQ  AFSC  planners  into 
the  mission  area  plans.  An  office  of 
primary  responsibility  at  HQ  AFSC  is 
assigned  tor  each  of  the  mission  area 
plans  This  grouping  by  mission  area 
integrates  the  separate  sub-mission 
analyses  into  cohesive  packages  that 
address  total  mission  needs  i Figure  3>. 

Plans  are  assembled  in  a  two-step 
process  First  planners  propose  a  can¬ 
didate  20-year  investment  program 
:r\u  is  not  fiscally  constrained.  The 
'ime-pnased.  total  estimated  cost  of  the- 
candidate  weapon  system  acquisition 
programs  normally  exceeds  the  pro¬ 
ve  ted  available  financial  resources. 

in  the  second  step,  plans  are 
tailored  ■  by  deleting  some  acquisition 
programs  and  delaying  the  starts  tor 
others  to  select  the  investment  pro¬ 
gram  that  will  best  meet  needs  within 
realistic  budget  constraints.  !n 
developing  a  Vanguard  fiscal  con- 
-* r.unt .  vve  look  back  20  years  at  those 
investment  funds  that  AFSC  executed 
to:  !!•••  users  I  h>  investment  funding 
averaged  1.2  percent  annual  real 
vow th  For  Vanguard,  then,  we  use  I 
percent  annual  real  growth  projet  ted 
trow  the  budgeted  (meal  year 
‘hri  'iigt'ou!  the  20 ear  period 

[ho  phase  tit  the  Vanguard  p roc  ess 

i  '  ills  ir'  consultation  with  the 
opera:  mg  I  ommantis  Cue  h  (  oordma - 
‘ ion  ensures  pians  address  the  highest 
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priority  goals,  overcome  mission  area 
deficiencies,  and  deliver  the  capability 
needed  to  satisfy  future  mission 
requirements. 

The  HQ  AFSC  development  plan¬ 
ners  brief  the  final,  constrained  mission 
area  plans  to  the  product  divisions  to 
ensure  the  plans  are  accurate, 
reasonable,  and  executable. 

Two  Types  of  Planning 

Kejimeiui’iif-.  Pull.  Vanguard 

doc  uments  the  requirements  pull  It 
responds  to  the  vang  c  ommands  pro 
tec  ted  requirements  by  focusing 
development  planning  on  the  most 
serious  dehc  icnc  ies  vvt  will  eru  ountei 
during  I  he  next  !  wo  dec  ades  I  he  mis 
sion  area  plans  serve  iwo  btoad 
purposes- 


-  They  provide  for  regular  and  formal 
coordination  of  needs  and  potential 
alternative  solutions  between  AFSC 
and  the  operating  commands. 
Operating  commands  requirements 
are  evaluated  in  the  context  of  Defense 
Department  and  Air  Statt  guidance. 
Alt  ernative  systems  to  satisfy 
operators  needs  are  analyzed  and 
comparatively  evaluated  with  the 
operators.  I  hev  identity  technologies 
which  must  be  matured  to  support 
future  preferred  systems  these  are 
provided  to  AFSC  laboratories  toi  use 
in  developing  tec  hnologu  a!  investment 
s!  rategv 
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Additional  \  anguard  List’s 

I  h«-  primary  (unction  ot  Vanguard 
:-  to  identify  possible  new  program 


'tails  toi  tnc  next  20  vcar**  to 
alleviate  dtto  it'tii  :v's  during  th.it 
per  iod  In  tlu  broader  c  ontext . 
\  anguard  also  highlight*-  potential  im- 
p.i  ts  on  new  systems  on  AFSC  s 
organizational  structure  and  test 
resource  needs. 

I  he  Air  Force  Systems  Command 
is  committed  to  executing  the  most  et- 
t u  ;eiv.  programs  possible.  One  way  to 
improve  etticiencv  is  to  exploit  the 
leverages  available  in  pre-tull-scale 
engineering  development:  studies  in¬ 
dicate  S5  percent  ot  the  total  program 
cost  is  locked  in  during  these  early  ac¬ 
tivities.  Early  cost  estimates  and  em¬ 
phasis  on  supportability  at  the  onset  ot 
program  formulation  provide  needed 
insight  tor  decision-makers.  Vanguard, 
through  the  examination  ot  alter¬ 
natives  is  one  method  used  by  AFSC 
tor  reduction  ot  risk  and  cost.  Also. 
AFSC.  is  working  closely  with  the  Air 
Force  Logistics  Command  to  incor¬ 
porate  logistic  considerations  up  tront 
and  early  on  in  the  system  planning 
process. 


Presentation 

Results  ot  the  \  anguard  analvsc-s  arc 
presented  in  two  wavs,  briefings  and 
the  Vanguard  Planning  Summurv 
report . 

An  executive-level  briefing  tor  eacn 
plan  is  presented  to  the  p  oduct  divi¬ 
sion  commanders.  Air  Force  rnuior 
commands,  the  HQ  l.  SAF  board  struc¬ 
ture.  and  industry. 

The  Vanguard  Planning  Summarv  is 
produced  annually  to  document  the 
mission  area  analyses.  It  is  a  com¬ 
prehensive  description  ot  a  recom¬ 
mended  research  and  development 
program  strategv  which  represents  the 
highest  potential  return  on  investment 
during  the  next  20  years.  The 
document  is  available  through  the 
Defense  Technical  Information  Center 
Building  5  Cameron  Station  Va. 
22314. ■ 
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elsewhere  Cine  ot  our  most  exciting 
and  promising  initiatives  Steinbach 
said  is  to  augment  the  Belvoir 
worklorcc  be  engaging  universitv 
research  teams  jn  collabora :  i  ve 
ettorts  with  out  scientists 

1  he  C  enter  established  a  l  niv  ersitv 
Rel.it ions  Ac t ion  Group  wine  h  lias  in¬ 
itiated  discussions  with  17  universities 


George  Mason  I'niversity  in  con 
nine  tion  with  the  Sue  iety  ot  I  ogistic  s 
F.ngineers  will  otter  continuing  educa¬ 
tion  courses  m  logistics  beginning  m 
September  and  c  ontmuing  thioughout 
tlie  academic  ve.u  All  c  i mrsc-s  evil  1  he 
1. n;i'!i!  a!  ! he  (  .forge  Mason  I  nivcl'i 
tv  1  ’r oli-ssiop.il  C  inlet  5401  North 
Hurt, ix  Drive  Arlington  \  a  Some 
ionises  whith  provide  b.ic  kground 
and  context  in  the  role  ot  logistics 
within  the  organization  have  been 
designed  | or  the  novice  in  'lie  field  C  )t 


throughout  the  Lnited  States  The 
groups  primary  task  is  to  relate  the 
Center's  tec  Finical  problems  to  resea  rc  h 
interests  o!  established  universitv 
teams.  A  kev  feature  is  emphasis  on 
collaboration  calling  tor  active  par¬ 
ticipation  by  Bel  voir  scientists 
graduate  students  and  t. unite 
members,  ■ 


Introduc  torv-leve!  courses  include 
tile  Integrated  logistic  Support  Pro¬ 
gram  1  ogistic  Support  Analvsis  and 
logistic  Support  Analvsis  Record 

■  1  S A  1  SAlx  Quantitative  Method' 

tor  1  ogistic s  .Hid  introduc  tion  to  (.  on 
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s  weapons  systems  become 
more  sottware  dependent 
weapon'-  system  acquisition 
managers  are  discovering 
more  frequently  that  thev 
are  sottware  acquisition  managers 
Ibis  article  contains  guidance  on  the 
management  ot  a  software  develop¬ 
ment  . 

Management  goals  of  the  software 
development  process  are  to  produce 
cost-effective,  reliable,  modifiable  and 
maintainable  software  consistent  with 
error  tolerances  ot  the  system.  The 
nature  ot  the  sottware  development 
process  provides  many  opportunities 
tor  a  misdirected  management  em¬ 
phasis  to  exceed  required  goals  in  one 
area  at  the  expense  ot  another.  For  ex¬ 
ample.  excessive  emphasis  on  sottware 
error  reduction  could  result  in  sottware 
that  is  well  below  the  error  tolerances 
ot  the  system  but  has  an  excessive 
development  cost  and  schedule.  In 
other  cases,  the  emphasis  could  be  on 
minimization  of  development  cost  and 
result  in  sottware  that  is  difficult  to 
maintain  during  the  lite  of  the  system. 
This  article  addresses  sottware 
development  methodologies,  manage¬ 
ment  issues,  risk  management,  and 
sources  ot  assistance  tor  achieving  the 
desired  sottware  acquisition  goals. 

Software  Development 
Methodologies 

There  are  many  software  develop¬ 
ment  methodologies  and  as  sottware 
development  tec  hnologv  evolves 
more  methodologies  are  likely  to  up 
pear  The  con  true  tor  s  c  hou  e  ot  sott 
ware  development  methodology 
depends  on  various  (actors  Some 
methodologies  work  better  with  cer¬ 
tain  types  of  applications  others  work 
better  w  it  h  c  ertain  languages  I  he  c  on 
t  rac  tor  s  ex  per  icm  e  with  t  he  part  i<  ular 
a  [’pile  at  ion  or  with  similar  appilc  at  oris 
can  influence  the  choice  ot  develop 
merit  methodology  While  military 
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software  development  standards 
generally  are  designed  around  top- 
down  development,  this  is  clearly  not 
the  best  choice  in  all  applications  and 
environments.  In  general,  the  procur¬ 
ing  agency  should  strive  not  to 
prescribe  a  specific  methodology  to  the 
contractor,  but  should  evaluate  the 
methodology  proposed  by  the  contrac¬ 
tor  to  ensure  it  is  an  effective,  uniform 
and  disciplined  approach  to  software 
development  resulting  in  sottware  that 
is  easily  supported  and  maintained.! 
Whichever  sottware  methodology  is  I 
chc'sen  there  must  be  a  consistent 
systematic  and  structured  approach  to 
deal  with  each  software  development 
activity  requirements  analysis  design, 
code,  integrate,  and  test.  The 
methodology  must  be  documented  and 
procedures  must  be  followed. 

All  well -struc  tured  development 
methodologies  will  include  several 
phases  ot  development.  The  six  phases 
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1  : i  rvrarc  Design  l  Vt  a  tied  IV'ig” 
l  odt  and  I'm:  I  es*  C  o::i;n;!fr  Sort 
ware  t  omponent  CSC  Integration 
,i:\i  t  omputer  Software  Configuration 
Item  v.  SC  1  Testing  The  phases  or 
,iri'  development  tan  occur  dur 
,?g  ar".  or  the  phases  ot  weapon1. 
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assets  to  evaluate  pre^rt-N.  a'.  .rare 
I  he  personnel  md  hardware  assets  tor 
contractor  ami  the  government  sup 
per:  organization.,  must  hi  monitored 
c  ontinualiv  during  soft  watt-  develop 


'men!  i  e  tor  a  varie- 

rru*nt  * 

o  ensure  that  asset  cjtnstraints 

new  software  tan  be 
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'tng  anv  phase  ot  a 

risk -retluc  tng  activities  sue  h  as  design 
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Ihf 

next  issue  addressed  is  the  ade 

—  .  dated  requirements  during  develop- 
rrr e r  •  or  inclusion  ot  a  1  re.uiv 
developed  software  as  p  irt  ot  a  new 
development  some  phases  tan  be  ab- 
bre\  i.iteti  or  non-existent  tor  portions 
ot  the  software  1  ikevvise  not  all  por- 
‘ : o n s  ot  software  development  need  be 
:n  the  same  development  phase  at  the 
same  time  Despite  the  need  tor  ties- 
delitv  in  the  phasing  ot  sort  ware 
development  eat  h  phase  must  be 
planned  and  managed  to  ensure  that 
ba'ii.  software  and  anv  motiitit  ations 
reuuired  to  the  hasp  software  .ire  ad 
dresseii  systematic ally 

Management  Issues  in  Software 
Development 

I  here  are  tour  fundamental  manage¬ 
ment  issues  to  be  addressed  in  the  early 
s’ages  ot  software  development 
(dener.iliv  they  are  addressed  in  anv 
rontraitors  proposal  but  must  be 
managed  and  monitored  continuallv  to 
ensure  safistat  to,  v  solutions  In  many 
ases  these  are  the  same  issues  a 

manager  would  ene  ounter  in  anv  at 

tp.;isi*  ion  task  mother  i  ases  the  issue 
is  pei  itu  to  the  tact  that  the  develop¬ 
ment  is  software 

The  first  issue  (overs  resource 
availabiiitv  and  organization  Not  only 
must  the  contractors  assets  he 
monitored  to  ensure  availabiiitv  ot  the 
proper  skill  and  expcrienc  e  level  to  per 
form  the  development  successfully  but 


quae  v  and  at  h le vabilit  v  ot  the 
development  schedule  and  milestones 
Milestones  must  be  quantitative 
measurable  events  trom  which 
reasonable  estimates  ot  the  develop¬ 
ment  progress  tan  be  ascertained  The 
schedule  must  provide  sufficient  time 
tor  the  contractor  to  perform  develop¬ 
ment  tasks  but  it  also  must  ensure  that 
the  government  has  sufficient  time  to 
perform  its  required  functions  such  as 
review  documents  or  evaluate  designs 
The  contract  schedule  must  avoid  in¬ 
advertent  costly  resource  fluctuations 
at  contractor  or  government  support 
organizations  through  arbitrary  com 
prcssion  ot  development  phases  or 
delays  between  development  phases 

The  third  issue  involves  data  rights 
and  documentation  requirements  ot 
the  software  In  situations  where  com¬ 
mercially  developed  software  is  used 
as  a  portion  ot  a  system  data  rights 
and  doc ument.it ion  available  tor  pur- 
c  hase  bv  the  government  tan  vary  bv 
component  within  the  total  software 
package  This  rapidly  can  become  a 
complex  issue  as  the  acquisition 
manager  tries  to  ensure  that  the 
government  acquires  needs  to  perform 
the  mission  during  the  life  ot  the 
system  This  t  an  result  in  increased  at  - 
quisition  tosts  i paving  tor  data  rights 
to  i  ommeri  i.il  dec  clop  men  t  or  paving 
tor  t  ustomizeil  development 1  in  order 
to  retime  lite  icile  costs  .'providing 


.  I  ■  . 


to:  t-as\  updtitt^ 

:",  the  future  Sinn'  the  pun  hast-  ot 
,:.iM  rights  tan  he  costlv  a  proper 
tc-  brut  a.  assessment  :;hm  be  made  bv 
qualified  personnel  regarding  what 
-fata  ngr.ts  art-  needed  based  or  a 
:ta:>l;t  assessment  ot  thf  amount  of 
software  maintenance  and  modifica¬ 
tion  expected  during  the  1 1 1 1 ■  ot  thf 
s\  sfe  ;n 

I  hf  rourth  issue  is  establishment  t't 
software  qualitv  f valuation  criteria 
hfw  problems  tan  disrupt  a  software 
development  program  like  a  discon - 
nett  between  the  procuring  agencv  or 
its  support  organizations  and  the  con¬ 
trol  tor  developing  software  concerning 
ev  aluation  criteria  tor  su  cesstul  pass¬ 
ing  ot  a  program  milestone.  These 
disagreements  could  concern  adequate 
ot  preliminary  documentation  submit¬ 
tals  developmental  tests  to  be  con¬ 
ducted  on  software  or  differences  in 
preferences  tor  coding  standards  or 
software  development  practices.  When 
disagreements  occur  the  procuring 
agency  must  evaluate  their  significance 
and  substance  in  terms  ot  program  im¬ 
pact  Does  the  disagreement  reflect 
only  personal  preferences  ot  the  par¬ 
ticipants’  There  are  ways  to  develop 
software  and  usually  a  variety  of 
ways  to  interpret  data  item  descrip¬ 
tions  Or  does  the  disagreement  reflec  t 
risk  to  the  program  in  the  quality  and 
completeness  of  the  software  being 
developed?  Not  an  easy  determination 
this  often  causes  things  to  be  fixed 
that  were  not  broken  a  surprise  to 
the  contractor  which  can  divert 
resources  from  the  real  task  resulting 
m  schedule  slippages  and  cost  over¬ 
runs  Sometimes  disagreements  can  be 
avoided  it  portions  of  early  design 
reviews  and  progress  meetings  are 
dedicated  to  getting  agreement  ot  the 
parties  on  templates  formats  and  ex¬ 
amples  ot  doc  uments  code  and  test 
pr>  h  edures 

Software  Standards  and  Data  Items 

irv  standards  i  an  be  acquist- 
aids  <o  trie  huver  ami  seller  of 
m.u’.irv  pn  sha  *' and  servi<  t-s  bn  the 
•  I  Mr .  standards  uientitv  re¬ 

el  .iremer*.  app.v  mg  fo  the  produt  t  or 
•erv  a  e  to-  •  h.e  huver  mihtarv  stall 
-lard--  represent  an  accessible 
M;h  Table  p.n  i>  ,ige  of  requirements  to 
it-  lev  .rd  on  a  seller  to  ensure  a  qua  • 
prod-.a  •  Hi-  ause  i<f  the  difti  retl * 
",  pi  s  o*  sot  t  w  ar  i  dr  v  eh  >pment  tor  cl  it 
terer*  applii  atloris  i  are  must  be  ou 


•  laivl.iM-  !  '"i  i-  ' > 't  n  " t-  .rp 
ptoai  n  to  appl at, on  or  »  m< 

O'  ’t-s  not  in  i  e  s-ar ,  ,v  rr'pr  >  -  v  t  .a, .'  v 
ot  :  he  sot: wan  ill  v  cm  -pc  -I  in.'  .  <  : 

t  a . i' . \  f  f 1 1 : *s  c  i  "s*  „r v.  ‘•I  f  reik.it  ’ii 

development  Appl ,,  a:  :■  -n  ot  ..nn» 
i  essat  v  r  eciui:  enients  i  an  sign  it  a  ar” .  v 
iru  rease  the  cost  or  sot ! w are  develop 
ment  without  tangible  bench'  to  the 
I  department  ot  IVteiise  Mihtarv  stan 
da  ids  must  be  tailored  tor  the  develop 
ment  to  avoid  wasting  ami  ted  m 
sources  ot  the  weapon  sv  stt  m  ,u  p 
tion  This  is  not  an  ease  ‘ass 


Acquisition  managers  are  supported 
by  specialists  m  each  area  ot  expertise 
logistics  documentation,  testing 
financial  management  contracting' 
who  may  not  have  the  larger  view  ot 
program  needs  Too  often,  when  an 
acquisition  manager  attempts  tic  tailor 
out  a  redundant  mihtarv  standard  re¬ 
quirement  from  the  program  a 
spec  lalist  involved  mav  perceive  this  as 
an  attack  on  his  specialtv  in  general 
This  results  m  an  overwhelming 
resistant  e  to  the  deletion  ot  the  require¬ 
ment  in  question  causing  the  acquisi¬ 
tion  manager  not  to  remove  the  re 
qmrement  I  he  result  is  that  mihtarv 
standards  seldom  are  adcquatclv 
tailored  tor  anv  partuuliar  software 
ac  cjuisition 

DOD-STD-21t>7.  I  his  stand.irc!  is  the 
cornerstone  ol  the  current  software 
standardization  effort  in  th«  Depart 
men!  ot  I  detense  1;  contains  n 
inurements  tor  the  development  ot 
Mission  l  ritual  Computer  ‘xvstem 
MC  t  S  sottwaic  It  contains  m 
quircments  that  <  an  be  applied  to  v  ir 
tuallv  all  tvpes  of  software  deveiop 
men*  but  i  otif  .iming  ah  oi  '  to  s.  n 
quncments  p  contains  more  nquuc 
merits  than  \\ou;,j  gc-nei.illv  It  it-  e- 


s.ii  \  on  anv  orie  software  dcvc.op 
ment  Significant  etfoit  must  be  c» 
pended  bv  uvntr.n  ting  agenc  a  s  to  en 
sure  that  thev  tailor  out  urmecessarv 
requirements  ot  the  standard  Some 
success  has  been  realized  in  this  utca 
bv  using  combined  government  indus 
trv  working  groups  to  set  guidelines 
tc>r  the  tailoring  ot  M !1  -S  I  [  d-2  1  oh  t or 
speutlc  tvpes  ol  software  develop 
ment 

MIL-STD-1521.  Ihw  standard  pre¬ 
scribe'-  requirements  lor  conduct  ot 
technical  reviews  and  audits  on 
systems  equipment  and  computer 
software  Again  in  trying  to  be  all 
things  to  all  programs  this  standard 
requires  tailoring  tor  any  partic  ular  ac  - 
cjuisition.  bor  software  acquisition 
there  is  room  tor  interpretation  in  the 
requirements  ot  this  standard.  Addi 
tional  specifics  generally  need  to  be 
added  to  clarity  requirements  tor  a 
particuliar  software  development 
those  avoiding  independent  interpreta¬ 
tions  and  misunderstandings  between 
contracting  agency  and  contractor 
regarding  what  is  required  and 
expec  ted 

\lIL-STI)-4s>0.  I  his  standard  cstab- 

lishcs  uniform  practices  lor  specific  a 
t ion  preparat ion  Software  ac  quisit ion 
managers  must  evaluate  flit  le  vel  ot 
spec  it  ic  at  ion  required  tor  their 
progt  ams 

Mil  -STD-483.  I  his  standard  estab¬ 
lishes  uniform  c  ont  igur.it  ion  manage- 
men;  requirements  that  must  be 
M m t ed  lot  ,m v  -pt -  .t ;t  -ot t w art 
Tl  tjUlsl  '  a 

Software'  Data  Item  Descriptions 

'  DlDse  I  lie  POD  SID  :icP  identities 
data  He  m-  that  c  an  be  tequued  on  anv 
si  ‘  w  a  r  c  .e  i ;  m  -  - '  a  m  !  '■  ev  !  a  h  n  I  o 

in  t  r  i  :i  e  '  c  and  t  <pt  :  a !  'o;  ■  ,i  m  ;  -  u  p 
pi  'i  t  s,  •(  r  -a  .i:  c  d.ii  a  it  i  "  -  a :  i  ,  .-i  .  i  1  -  ■ : 


’•''ft  : v. : p. »sfs  t  cn^urt’  ^oPtuuNH 
-  a  a«  tig  appropriate  progress  on  the 
xoitwate  decelopment  and  w  maintjin- 
t’u  v  i schedule  control  2  ensure 
a  :  ;n  !,i:  ion  ot  tec  h n iv  al  pertormanc  e 
•cut..  remi-nt'  b\  software  dec  eloper 
art  v  .insistent  ••Mt:-.  interpretation  in 
tet'cted  proe-..!-.ng  agenc \  and  •  3 
i  mi  p;  o’  -l:! tit.  lent  information 
at-.-.:-  so  that  it  .an  he  rrtain- 

"a::-.  a,.:.:'.:  x\xtctn  ! :  1 1  -  l  \  t  It  lath 

'i’I'a  ja  a. q wt  ion  itunaitin  must 
, : i  ‘ a  nal  .titormatton  :e 
to:  : re  program  ia.h  ot  the 
a:'o'. .  iti.i.  atot  At'.;.!'  teqatte  onl\ 
■"a1  ft  o:  mat  :o:i  [  nru-c  exxurv  data 
-  I"'  o',  tt'c  it.'vitt'ttH'ti:  motif', 

t't .  a..'<  t ft  .I'tva.  to:  must  prepare 
a .  require  go-.  t- rn tut  n t 

1 1  " ■  , ; .  i  -*  !  i '  !  t  v ; ii\\  *» p t W i '*s»i i ' P t  It  Is 

r  t  v  ..  .1? ::rp*'rt  jn?  U'r  sotiwjrt*  .u 
:  s  p  i  rvv  ;rv\  pm u.f  <.  p** 
.o.  to-  f.  a!  ..at .nit  submitted  data 

ot”'.  f.pf.oi.o.  tt'oxf  tor  f  \  aiitattng 
t”t  t't  pe t :  o:  ma tii  e  ot1  require- 
■"t  t".  :■  ‘ t  •  pt t- : a:  ton  It  : net t  >  no  ore 
■  I  a  a  a  -o.t'tt’lttf.!  item  propt-rK 
at'.:  o  ,  -  a  t't .  i".ir  it  ton  to  begin  tv;  it' 
the  p-ovra't'  has  a  significant  risk 

tb.r  tin  software  acquisition 
manager  detmex  information  re 
.  rt  'ttif  t’t  x  nt't,fxxar\  tt'r  software 
.tfVf !. 'p.mt  nt  there  tx  often  an  oppor- 
■  , n ; : tor  .oxt  savings  in  thf  atquisi 
ttoti  C  oxt  savings  tan  hi-  realized  bv 
authorizing  vendor  tarm.i!  tor  xub- 
rttxxion  ot  data  itemx.  T  his  alltnvx  the 
ontrattor  *o  submit  information  in 
vhatf vt-r  tormat  in  use  tt'r  rn.in.wt" 
nett-  ot  the  t  ontrat  *  witht'ut  having 
o  ri'tormat  tlata  tt'  xpetilit  data  item 
ifxt  ription  Benefits  t't  usings  this 
t-t  hniquc  must  he  evaluated  however 
W.unst  the  additional  review  prt'cesx 
hat  mav  he  required  by  the  procuring 
wftit  \  to  glean  inti'rmation  desired 
r.'tt'  an  unfamiliar  tormat. 

Risk  Management  in  Software 
Development 

I  here  are  four  basil  tvpex  of  rok  in 
•he  tleveli'pmen!  ot  software 

I  Software  does  not  meet  re 
quirementx.  Basic  requirements  tor  the- 
pertormante  ot  the  svstem  must  f'e 
’tanxlatetl  into  a  sot t ware  xpeutua 
•a  t'  I  his  specification  must  then  he 
translated  into  operating  t  omputer 
i  ode  Fiat  h  ot  these  translations  pro 
■.  ides  xignilu  ant  .  otmiiunn  alien,  dit 
tu  ulties  in  that  terminologies  used  bv 
the  persons  tfi.it  uientitv  requirements 


are  probable  no!  .hath,  understood  he 
persons  who  write  .  omputer  . . ide  ami 
vue  versa  Persons  involved  in 
translating  the  requirements  into  a 
sottware  spetituation  must  F'e  sutt: 
c  ienti\’  tamiliar  with  svstem  opera 
tional  requirements  and  with  the  pro 
c ess  ot  developing  and  executing  sot! 
ware  to  ensure  there  are  no  significant 
.onimunnation  dittieulties  in  this  pro 
less  Program  managers  must  ensure 
there  are  xuttiuent  technical  reviews 
during  software  development  tor  earlv 
identituation  and  eorreetion  ot  com 
munuation  dittieulties. 


2  Cost  Schedule  overruns  com 
mon  in  software  development  oum 
due  to  the  innate  clittu ultv  in  software 
development  ot  aiaurateh  measuring 
work  accomplished  anil  estimating 
work  remaining  Measuring  progress 
in  software  development  is  analogous 
to  measuring  progress  in  a  golt  game 
On  a  300-vard  hole  the  Simt  drive  is 
2.40  v arils  The  see ond  shot  art i vex  on 
the  green  Is  this  hole  2  1  complete 
halt  complete-  or  are  tour  putts  .. 
retaining’  in  software  development  it 
is  at  least  this  hard  to  estimate  pro 
gress  Many  measures  ot  progress  used 
in  t Fie  past  sue  h  as  numt'ei  . >t  lines  ot 
.ode  finished  vs  number  ot  lines 
estimated  tor  the  total  svstem1  have 
proved  to  be  unsu. .  exxtui  in  estim.i’ing 
the  a.  dial  amount  ot  work  remaining 
in  a  sottware  development  I  he  most 


su.iesxtul  tec  hr  'eji.es  used  tot  tr.tas.it 
in g  progress  have  involved  the  analvsis 
ot  sottware  documentation  submittals 
.is  data  items1  during  the  development 
.  v.  It- 

'  Resulting  sottware  has  biu.-tct 
than  expected  lite-cvcle  cos;  I  has 
situation  can  arise  e\  :  when  software 
dev elopmcnt  meets  system  re- 
c]uirements  and  w  performed  within 
e.'st  .md  s.  hedule  constraints  I  his  is 
usual!-,  a  manifestation  ot  dclu  it  nc  it-s 
:n  the  sottware  development  ptoc  t-ss 
Be.  ause  s.'ttware  is  essential!',  a  retie. 
tH'n  ot  a  s.'ttware  engineers  though,: 
pro.  t-ss  m  i  real  mg  a  solution  to  a  pro 
idem  it  is  possible  tFi.it  attet  sottware 
|x  developed  It  will  F’e  dlttu  lilt  .'I  ml 
possible  tot  another  person  to  recon 
struct  (has  thought  ['roc t-ss  to  modi!-, 
maintain  or  correct  the  resulting  com- 
putei  .ode  This  often  is  caused  !n 
cither  inadequate  constraints  on  sot! 
ware  engineers  or  programmer'  in  t hi 
use  ot  techniques  tor  manifesting  a 
thought  process  in  computer  code 
coding  standards  or  bv  inadequate 
s.'ttware  documentation  Note  that 
good  coding  standard'  and  practices 
are  more  important  in  reducing  life 
i  v.  It-  c  osts  than  eve  c-ssive  doc  u  merit  a 
tion  It  is  easiei  to  maintain  good  well 
stiu.  fuii-.l  code  v\  1 1  h  minimal 
ilouiitu-n  tation  tfian  to  maintain  i'.id 
iode  with  great  dot  umentat ion 

4  Irisuttu  ic-nt  F’asehne  control 
which  can  jru  iude  evolving  re¬ 
quirements  and  changes  m  tec  Finical 
data  during  sottware  developmc-nt  can 
h a \  >  mipac !'  permeating  the  sottware 
development  Interlaces  designed 
do.  umentated  and  reviewed  tan  F'e 
.  hanged  draxticali'.  i'\  things  that  ap 
peat  on  tile  surface  to  have  negiigi- 
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.in  or  prrtorm-N  ditto! v:Ui\ 

:  ban  t  v  pee  ted  It  .an  in.  rease  lite-c  ve  ie 
costs  when  doc  lamentation  delivered 
t.ir’a  in  the  program  Joe--  not  reflect 
software  delivered  later 

Note  that  unless  the  software  ac¬ 
quisition  manager  is  familiar  with  the 
spec  :ta  tvpe  ot  software  development 
being  managed  it  is  usualiv  impossi¬ 
ble  to  assess  accurately  the  tour  risk 
;  ategories  l  epert  assistance  must  be 
acquired  either  through  speeitie  addi¬ 
tion  to  the  acquisition  manager  s  im¬ 
mediate  stall  or  bv  using  a  support 
! ’t  gan  :/at  ion 

Sources  of  Assistance 

I  wo  ",  pt  s  ot  g.  'vernttH  :•  t  -  «rgursi/u- 
■  :  o  n  s  a  re  a  v  a  ;  i  a  bi  e  to  provide 
i"  'tat',  e  to  sottwate  a.  question 
manage: s  depending  upon  their  needs 
!  he-,  an  the  defense  colleges  and  the 
Contra,  t  Administration  Service 
k  As  I  he  defense  colleges  like  the 
Nav  a!  Postgraduate  School  or  the  He¬ 
lens.  S',  stems  Management  C  ollege 
too.  research  resources  available  to 
ass;.;  ,n  investigating  spec  itic  software 
development  questions.  They  have 
assisted  :r  estimating  the  size  ot  sott- 
w are  proiec  ts  and  evaluating  software 
development  methodologies  There  are 
several  CAS  organizations.  AfT’KO 
Air  ho  rue  ARMPRO  Armvo 
H C  A S M A  IX  ASTRO  Hetense 
logistic  Agency  and  NAVPRO 
Navy  .  They  can  provide  assistance  to 
software  acquisition  managers  through 
out  the  life  cycle  ot  a  software  system  . 
The  CAS  mission  normally  starts  after 
the  award  ot  a  contract  but.  as  part  ot 
its  stated  mission,  can  perform  pre- 
award  surveys  ot  prospective  otterors 
Thw  helps  the  procuring  agency  ensure 
these  otterors  are  capable  ot  meeting 
subsequent  contractual  obligations. 
I  he  pre-award  survey  evaluates  ot¬ 
terors  management  and  technical  ex¬ 
pertise  financial  posture,  and  reviews 
their  records  ot  performances.  1  he 
C  AS  involvement  after  t  onlract  award 
■.enters  around  its  surveillance  and 
evaluation  ot  the  contractors  com- 
plianc  e  with  the  terms  and  c  onditinns 


l  A"  ot!  -  >  a 
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lie  pun  \ 

shou.d  re'- tew  available  l.  As  cupab: 
lit  i.  s  w  hen  e\  ah. at  in.:  . >pt tons 

In  addit.ou  th.e  pro.  urmg  aceru  ■. 
cat'  use  anothet  central  'oi  to  o verse, 
iht  contralto:  .!«  \  i  lop.iv  the  sott¬ 
wate  1  ho  cut:  in  an  independent 
validation  arid  verification  1\  Ac  \ 
mnlta.t  or  other  !  \  pes  ot  m.ppori  con 
tract  This  appro.u  h  has  risk  s  but  m,a\ 
constitute  the  otilv  soutce  ot  eiualt! :eci 
technical  assistance  to  the  sottwate  a. 
qutsiiton  manager  Risk  s  to  In  awat. 
ot  and  managed  me  ii.de 

Insure  the  reviewer  uetaailv 
providing  qualified  personnel  to  pit 
form  an  appropriate  technical  review 
c>t  the  specific  type  ot  software  under 
development.  It  is  not  unknown  tor 
contractors  to  use  the  review  ot 
another  program  as  a  training  ground 
tor  their  personnel  It  is  not  unknown 
tor  a  contractor  to  use  these  review 
activities  as  a  government  landed 
training  program  to  obtain  insight  in¬ 
to  the  techniques  employed  by  other 
contractors  to  establish  a  competitive 
e\lge  in  a  field 

-  [insure  that  obiee  turns  and  com¬ 
ments  raised  by  the  reviewing  contra, 
te'r  involve  substantive-  issues  ae  tual.'v 
reflecting  risk  to  the  program  rather 
than  another  agenda  ot  the  reviewing 
ce>ntracte'r  sue  h  as  ensuring  te'llow -e'n 
work  or  making  a  name  in  the1  tield  ol 
se'ttvvare  review. 

With  this  in  mine)  using  quaiitied 
contracted  support  ean  be-  a  cost- 
e'ttective  manageme'nt  te'e>l  in  the-  ac¬ 
quisition  c't  good  se'ttvvare  ■ 
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